PYTHON – Script to download youtube videos for offline viewing

I was interested in viewing this video of a news conference (USENIX 2016) on my trip home on Metro North Train, NYC => CT. The trip is about an hour an 10 minutes from Manhattan’s Grand Central Terminal to Milford CT, express train that is. My concern was that I would have choppy internet service on the way since I recently updated my laptop and the built in Verizon Mobile card was not activated yet. I would need to use my ATT iPhone as a hotspot, which proved to be very shakey at times. A colleague of mine recommended a website for making youtube videos available for offline viewing. The name of this site was:

http://www.keepvid.com

Right off the rip I was concerned that this site was infested with malware and any other bullshit associated with a free video ripping service. I used the site and was able to create a download of the video I was interested in, however who knows how sick my Windows based machine just got. I could of contracted anything from this site.

I thought about this and said, there has to be a better way, or a python lib for this, and low and behold a search came up with PYTUBE:
https://github.com/nficano/pytube

This library had some interesting features and literally blew away the keepvid site in regards to flexibility. Here is some explaining of what this library can do. Please have a look at the examples below, I will do my best to narrate them.

Here I use PIP to install the PYTUBE lib, you can ignore the DEPRECATION: warning for my outdated python that blares at you for being such an idiot.

Next up you can see that I am setting a variable yt(this is the video you want to download). Using python’s Pretty Print Lib you can run the pprint(yt.get_videos() method to see what formats are available for download.

Please have a look at the comments in the code for a bit more details in regards to what is going on, in this example I am using the filename Pulp_Fiction.mp4 for my filename I want to be when downloaded.

Ok so here is what it looks like when you execute the program:

As you can see we have a new filename with the video we asked for to watch without a streaming internet connection, here is a ls to show:

As always, I am sure there are better ways to do this and I am sure there is cleaner code. Most of this code was taken right from the authors site who is a badass, here is his link:

https://github.com/nficano/pytube

Hope you liked,
J$0N

Daily Weather Email – PYTHON – CRON – API

I was interested in the weatherunderground API for getting real time weather updates via the API and python.
WeatherUnderGroundApiLink:
http://www.wunderground.com/weather/api/
I was particularly interested in learning how to use the JSON python module. So I thought I would give you people a quick and dirty write up of how I get a daily weather update with endless categories via email on linux using python.

Ok so here we go, don’t get all excited and post this on hacker news and crash my t1.micro at AWS.

Let’s have a lookie at the python script where the magic takes place. O Yeah you will need to get an API KEY from WeatherUnderground for this. Look here:
http://www.wunderground.com/weather/api/d/login.html

The API will look like this once you obtain your copy:

Here is the link for requests:
http://docs.python-requests.org/en/master/
Ok let me break down the PY code:

It’s probably important to see what we get back from the API in JSON so the below code makes sense. Here we go, this is just pretty printing the JSON using pythons PPRINT module, Showing this snippet first to help understand the code.

SCRIPT:

OUTPUT:

Ok let’s parse this JSON and send a freaking daily email with the fields we want, O Yeah.

Here is the script commenting on what we are doing.

Here is what comes out of this python script when we call the JSON variables we are attracted to:

Ok let’s wrap this biatch into a shell script and have it call mail on linux so I can get a daily updates on the weather for Milford CT whenever I call the cron.

Here is the directory structure I setup for the scripts:

Here is the bash script that I wrote to bring it all together, it calls the python script and writes a log and mails it.

Here is the cron:

Here is my morning email message:

View post on imgur.com

Hope you had fun, I did. O and let me know how bad this post is and how you can do it better.
Love you

Analytic’s – Fuckin A

Well I thought I would write up a quick post to demonstrate that even the slower kids like myself can achieve web visitors hitting their site if they put some effort behind it. I started this blog back in late 2012 and I only posted a couple of code snippets here and there. Then my coding skills developed a bit and I kept the domain alive so it started to get a bit more traffic. Anyway this really is not much traffic, but it’s neat to say that I have had my blog up and recording visits for some time now.

Google Analytics JR.org

So here it is, if you want to advertise hit me up at [email protected] 🙂

CYGWIN – clear.exe from scratch C Program

Hello all,

TL;DR – I wrote a C program to use as clear.exe on cygwin, I know about CTRL-L but I wanted a binary for scripts, and I am used to typing clear at a terminal

I was recently enforced to use Windows as my daily computer at work for all types of compliance PCI reasons, reasons that I do not wish to dive into. However I am now on Windows, I spend most of my day in a BASH shell on a remote system, so it’s not that bad. However traversing around Windows with CMD or PS sucked. I was introduced to cygwin and I was really impressed, I was able to use a ton of tools that I was already familiar with on the Linux CLI. Native SSH,SCP were huge for me at this time, also the ability to customize the colors with the TTY settings was awesome.

One issue I ran into was I liked to use the clear command in my scripts and from the CLI when I want a fresh terminal. Well, I figured I could just re launch the cygwin installer and search for clear.exe and install it. This was not the case, unfortunately I was instructed to install ncurses and the associated libraries. I followed these recommendations and never got clear.exe to work.

So I did what any computer scientist would do, research how to write a C program on this and compile it and copy it to my $PATH and wouldn’t you know it works!!.

So Here it is in all it’s glory. clear.exe for Windows and CYGWIN:

### Clear no workie ###

Ok so we can see the clear program was not working in the above example. So I wrote the following lines of C code below and compiled them.

OK, cool, now you will need to get GCC on your Windows machine from the CYGWIN installer, so just re run the installer and search GCC, you can install it from there like so.

View post on imgur.com

Ok now for the awesome stuff, it’s time to compile this with GCC and create a new clear.exe binary for Windows and CYGWIN.

Now that we see we can use the new executable to clear out cygwin shell, let’s copy it somewhere so we can just type “clear” to get what we want.

And there you have it, have fun!!!

Jason

BASH Clear Semaphores Linux

Rsync Market data from Multicast Collector To SAN – BASH

Here is a snippet from the log:

FInd Large Files Specified By Size Linux – BASH

You can execute this and get your results to stdout.

Examine Log File To Be Sure O values Are Not Recieved – BASH

Automatically Restart Linux Application With All Arguments KSH script

Here is a script that you can use to start your application on Linux. What this script will do is read in all the arguments from the command line and store them in a special shell variable. If the application segmentation faults, the script will automatically start the application using all the command line arguments. Also this will write that the application has died to any syslog on the system.

Here is how it should be called.

And here is the actual code written in KSH.

Linux Restart Process / Application When Memory Threshold Is Reached

Here I needed to kill a c++ process that was leaking memory in order to save the server from meltdown. Very simple while loop that calls the external command “free” and parses it’s output with awk to store a variable. Pretty neat and it works.

Jason