Tutorial: A Slick Way to SSH into your AWS EC2 Instance [Ubuntu/Cygwin]

So I found out there are some tricks to both getting SSH into your ec2 instance to work, as well as getting it to work in a slick manner. My goal is for you to be able to simply type:

aws_ssh

into your terminal, and for that to magically work! Only the first two steps are really relevant in getting it all working, but the next few make it slick and AWSome...

Here's what I'm going to show you:

  1. Set up proper permissions on your .pem file
  2. Test SSH in Terminal
  3. Make an executable file to SSH
  4. Create a shortcut to that SSH

SET UP PROPER PERMISSIONS ON YOUR .PEM FILE

So the way AWS sets it up is that if you try to SSH into your instance with a badly permissioned key (.pem) file then it will send some message like "ZOMG we need your key to be more secure, permission denied, public key" (btw that was totally paraphrased). But anyways, what you need to do is simply change permissions on the file to something a little more secure (like owner read-only) and AWS will be a little less temperamental.

sudo chmod 600 /path/to/key.pem

BOOM. Now that should be it for that.

 

TEST SSH IN TERMINAL

This is just a basic test to make sure that your .pem file is up-to-par now and everything is working. So go ahead and try this out:

ssh -i /path/to/key.pem ubuntu@your.aws.public.dns

This should work! If it does sir/ma'am then proceed. If not, it may be one of a few things.

  • Double-check your ec2 public dns is right, for some reason Amazon has changed mine up once or twice...
  • Make sure you typed ubuntu@ in front of your DNS if you're using an ubuntu instance. I believe that a CentOS instance uses root@, and I have no idea for the others...

 

MAKE AN EXECUTABLE FILE TO SSH

So now that you've made sure SSH'ing into your ec2 instance actually works, let's start doing some magic. The first thing you'll have to do is create the executable file and then make it an executable file.

Create the executable file

Create that file somewhere (I'm calling mine aws_ssh.sh. Name yours w/e but just make sure it ends with .sh) and make sure you add the hash-bang so it knows to use bash and also include the ssh -i line that worked in the last step.

Make it an executable file

 To make this executable it's pretty easy... just do the following inside the directory that holds aws_ssh.sh.

chmod u+x aws_ssh.sh

Now you should be ready to go with the executable file! To double check that the file works just type this in Terminal:

/full/path/to/aws_ssh.sh

 

MAGICALLY CREATE THE SHORTCUT TO SSH

This next part is magical (for noobs like myself)! We are going to create a magical short-cut for you to easily ssh into your instance. Now that you know how to ssh from Terminal, and can do it all through an executable file (so you never have to memorize the DNS), we're going to create a simple one-worded shortcut to launch that executable from your Terminal!!

Essentially we're just going to create an alias from a keyword to that executable file (woops, did I just ruin the magic part?).

Here's how we do it:

Open ~/.bashrc and add the following line to the bottom of the file somewhere:

Just replace aws_ssh with whatever keyword you'd like to assign and you should be good! Just make sure you don't add any extra spaces or it won't work. To test this out EXIT TERMINAL, re-open it and test out your keyword. Type aws_ssh (or w/e you chose) and hit enter and you should have success!

 

As always if you guys run into any issues feel free to http://wesleyzhao.com/pages/contact me!

Python Web-page Scraping - Installing lxml and Beautiful Soup

So I've always used RegEx to scrape all my data. In fact, it can get pretty tough/tedious for a noob like me. I've been able to use it, but it's just a hassle. And until a few days ago, I thought this was my only route.

Fortunately for me, a few super-smart-engineer-entrepreneur friends (Noah Ready-Campbell and Calvin Young) told me about lxml and Beautiful Soup. They said it was a little tricky to install, but I didn't believe them... I tried it out for myself and actually had a lot of trouble getting it going. Eventually I stumbled upon something that made it pretty easy for me, but I'm hoping to turn that around and make it even easier for you to get.

So here it goes (disclosure: this worked on my Ubuntu EC2 emi and Ubuntu home machine):

How To Install lxml:

UPDATE: 

Try:

sudo apt-get install python-lxml python-beautifulsoup

Thanks to lamby of HN for this! I just tried it and it worked on my new Ubuntu EC2 ami...if anyone finds out this doesn't work please report it to me/someone-actually-important!

/UPDATE

The problem people usually have is there are just a lot of dependencies and it just seems that it never works. So here is what we'll end up getting through this:

  • libxml2 - the lxml library
  • libxslt1.1 - some other library that is a dependency
  • libxml2-dev - the libxml dev header
  • libxslt1-dev - the libxslt dev header
  • python-libxml2 - python bindings for libxml2
  • python-libxslt1 - python bindings for libxslt1
  • python-dev - python dev headers
  • python-setuptools - the thing that lets you run easy_install

So here's how it should all look like:

And if some of these exact commands don't work try searching for the package or updating your package directory:

 

Boom! Should be done. If you guys are running Ubuntu and have issues with this feel free to email me: wesley.zhao@gmail.com

 

How To Install BeautifulSoup:

This is much easier and hardly needs any instruction.

First go here to find the file you want: Beautiful Soup Downloads

Then depending on your file path and download you choose this is how you do it:

 

Once you have these installed check this post out: lxml; an underappreciated web scraping library.

The post has some great examples of how easy it is to scrape with lxml and BeautifulSoup. It's practically like being able to grab CSS tags!

Again if anyone has any questions feel free to email me! I know the set-up process can be a huge pain so...yeah.