Tutorial: A Slick Way to SSH into your AWS EC2 Instance [Ubuntu/Cygwin]

So I found out there are some tricks to both getting SSH into your ec2 instance to work, as well as getting it to work in a slick manner. My goal is for you to be able to simply type:

aws_ssh

into your terminal, and for that to magically work! Only the first two steps are really relevant in getting it all working, but the next few make it slick and AWSome...

Here's what I'm going to show you:

  1. Set up proper permissions on your .pem file
  2. Test SSH in Terminal
  3. Make an executable file to SSH
  4. Create a shortcut to that SSH

SET UP PROPER PERMISSIONS ON YOUR .PEM FILE

So the way AWS sets it up is that if you try to SSH into your instance with a badly permissioned key (.pem) file then it will send some message like "ZOMG we need your key to be more secure, permission denied, public key" (btw that was totally paraphrased). But anyways, what you need to do is simply change permissions on the file to something a little more secure (like owner read-only) and AWS will be a little less temperamental.

sudo chmod 600 /path/to/key.pem

BOOM. Now that should be it for that.

 

TEST SSH IN TERMINAL

This is just a basic test to make sure that your .pem file is up-to-par now and everything is working. So go ahead and try this out:

ssh -i /path/to/key.pem ubuntu@your.aws.public.dns

This should work! If it does sir/ma'am then proceed. If not, it may be one of a few things.

  • Double-check your ec2 public dns is right, for some reason Amazon has changed mine up once or twice...
  • Make sure you typed ubuntu@ in front of your DNS if you're using an ubuntu instance. I believe that a CentOS instance uses root@, and I have no idea for the others...

 

MAKE AN EXECUTABLE FILE TO SSH

So now that you've made sure SSH'ing into your ec2 instance actually works, let's start doing some magic. The first thing you'll have to do is create the executable file and then make it an executable file.

Create the executable file

Create that file somewhere (I'm calling mine aws_ssh.sh. Name yours w/e but just make sure it ends with .sh) and make sure you add the hash-bang so it knows to use bash and also include the ssh -i line that worked in the last step.

Make it an executable file

 To make this executable it's pretty easy... just do the following inside the directory that holds aws_ssh.sh.

chmod u+x aws_ssh.sh

Now you should be ready to go with the executable file! To double check that the file works just type this in Terminal:

/full/path/to/aws_ssh.sh

 

MAGICALLY CREATE THE SHORTCUT TO SSH

This next part is magical (for noobs like myself)! We are going to create a magical short-cut for you to easily ssh into your instance. Now that you know how to ssh from Terminal, and can do it all through an executable file (so you never have to memorize the DNS), we're going to create a simple one-worded shortcut to launch that executable from your Terminal!!

Essentially we're just going to create an alias from a keyword to that executable file (woops, did I just ruin the magic part?).

Here's how we do it:

Open ~/.bashrc and add the following line to the bottom of the file somewhere:

Just replace aws_ssh with whatever keyword you'd like to assign and you should be good! Just make sure you don't add any extra spaces or it won't work. To test this out EXIT TERMINAL, re-open it and test out your keyword. Type aws_ssh (or w/e you chose) and hit enter and you should have success!

 

As always if you guys run into any issues feel free to http://wesleyzhao.com/pages/contact me!

Tutorial: How to Set Up Dedicated MySQL Database Server Using Amazon EC2

DISCLAIMER: This is my current setup. Users with other distros beware.

So in the spirit of learning about scalability (and also after running into some issues dealing with several databases running on different local development environments we are using) I decided it would be a good idea to set up an EC2 instance solely as a database server. This tutorial will use MySQL as the database of choice not necessarily because I think it's the best, but because it is what I'm used to (I know... I'm lazy). So here's what we need to do.

  1. Assume you have an EC2 instance running for this
  2. Install mysql, apache2, php5, and phpmyadmin
  3. Configure your MySQL
  4. Configure your EC2 instance

ASSUME YOU HAVE AN EC2 INSTANCE RUNNING FOR THIS

Done. 

(JK, kinda. Here's a tutorial for this. Also I currently use the AMI mentioned above in the disclaimer. BTW You will need to SSH into your EC2 instance. Amazon has a pretty good tutorial if you just right click your instance in the management console, and click Connect).

INSTALL MYSQL, APACHE2, PHP5, PHPMYADMIN

Alright, now assuming we are running a clean ec2 instance, here are some of the dependencies you'll need. But before you do anything, just make sure your packages are up to date with the following code in your terminal.

sudo apt-get update

Install Apache (ignore this if you don't plan on using phpmyadmin or another front-facing MySQL app)

sudo apt-get install apache2

That was easy.

Install MySQL

 

sudo apt-get install mysql-server

During the install it might ask you if you want to preconfigure this for Apache. I almost got confused here, but click the Space Button to select, then Tab to hit "OK."

Oh, also, make sure you fill in a password for root (the more secure the better...but write it down).

Install PHP (ignore if you don't need phpmyadmin)

sudo apt-get install php5 sudo apt-get install php5-mysql sudo apt-get install libapache2-mod-php5

To double check everything is working by going into /var/www and adding a php file and try loading it. You may need to restart the apache2 server to check this by doing:

sudo service apache2 restart

Install PhpMyAdmin (not required if you don't think you'll need this)

sudo apt-get install phpmyadmin

Boom! So far so good....

Now try going to the webpage by typing: http://your.ec2.public.dns/phpmyadmin

Cool if it worked. If not bummer :(. Sorry, try again? 

 

CONFIGURE YOUR MYSQL

These next parts are not for the weak....also on a more serious note, these instructions may be specific to my Meerkat Ubuntu server.

But ah yes, what you'll need to do to configure your MySQL is two-fold.

Allow access from an outside IP

You'll need to navigate to your my.cnf file by typing:

sudo emacs /etc/mysql/my.cnf

Then change bind-address from 127.0.0.1 to 0.0.0.0. For me this was on line 52 and should look like this.

bind-address                 = 0.0.0.0

Grant MySQL user permission access from outside IP

Note: replace root with the user you chose (default is root). Replace securepassword with your password. You can change % if you want, as it is, it will allow access from any ip address.

Now you're at the final step my friend...

 

CONFIGURE YOUR EC2 INSTANCE

All you'll really need to do here is edit the security group of your instance to allow for the MySQL default pport (3306), and the HTTP port (80, if you plan to use phpmyadmin or something like that). 

To do this:

>go to the AWS Management Console

>Navigate to your EC2 Instance, and locate the name under the "Security Group" column

>Click Security Groups on the left bar, and then click the name of the security group assigned to your instance

>Click Inbound on the bottom

>Under Create a rule use the dropdown menu to find MYSQL and HTTP and add those both one at a time

>Make sure to click Apply Rule Changes or it won't save.

 

YEEE. Now you shall be ready to go. This will allow you to have a dedicated EC2 instance for all your MySQL database requests. 

Here are some of the tutorials I used to get this info: installing it all and configuring MySQL.

Python Web-page Scraping - Installing lxml and Beautiful Soup

So I've always used RegEx to scrape all my data. In fact, it can get pretty tough/tedious for a noob like me. I've been able to use it, but it's just a hassle. And until a few days ago, I thought this was my only route.

Fortunately for me, a few super-smart-engineer-entrepreneur friends (Noah Ready-Campbell and Calvin Young) told me about lxml and Beautiful Soup. They said it was a little tricky to install, but I didn't believe them... I tried it out for myself and actually had a lot of trouble getting it going. Eventually I stumbled upon something that made it pretty easy for me, but I'm hoping to turn that around and make it even easier for you to get.

So here it goes (disclosure: this worked on my Ubuntu EC2 emi and Ubuntu home machine):

How To Install lxml:

UPDATE: 

Try:

sudo apt-get install python-lxml python-beautifulsoup

Thanks to lamby of HN for this! I just tried it and it worked on my new Ubuntu EC2 ami...if anyone finds out this doesn't work please report it to me/someone-actually-important!

/UPDATE

The problem people usually have is there are just a lot of dependencies and it just seems that it never works. So here is what we'll end up getting through this:

  • libxml2 - the lxml library
  • libxslt1.1 - some other library that is a dependency
  • libxml2-dev - the libxml dev header
  • libxslt1-dev - the libxslt dev header
  • python-libxml2 - python bindings for libxml2
  • python-libxslt1 - python bindings for libxslt1
  • python-dev - python dev headers
  • python-setuptools - the thing that lets you run easy_install

So here's how it should all look like:

And if some of these exact commands don't work try searching for the package or updating your package directory:

 

Boom! Should be done. If you guys are running Ubuntu and have issues with this feel free to email me: wesley.zhao@gmail.com

 

How To Install BeautifulSoup:

This is much easier and hardly needs any instruction.

First go here to find the file you want: Beautiful Soup Downloads

Then depending on your file path and download you choose this is how you do it:

 

Once you have these installed check this post out: lxml; an underappreciated web scraping library.

The post has some great examples of how easy it is to scrape with lxml and BeautifulSoup. It's practically like being able to grab CSS tags!

Again if anyone has any questions feel free to email me! I know the set-up process can be a huge pain so...yeah.