Tutorial wrap-up

This is partially for my own sanity (as are most of my posts) and partially for the sanity of you readers (thank you to the two or three of you out there *smiley face*). Instead of having to traverse my blog to find all the neat how-to's I figure I'll compile a list here to make it easy.

I will be separating this into three categories:

  1. Hardy hand-holding tutorials - ones where I do my best to go step by step through the process with you.
  2. Read-between-the-lines tutorials - ones where I am still pretty detailed but may rely a little more on outside links/tutorials.
  3. Wesley-approved external tutorials - ones where I may just throw a link out there with a little description but I have followed the tutorials successfully.

I use tutorials I find on blogs all the time so I know how hard it can be to follow them, let alone find one that actually results in what you want. So believe you me that I do my best to make mine readable and fool-proof. In addition, any tutorials I link to I am confident that they work and are easy-to-follow (and if not then I will usually clarify where they go wrong or what details they may be missing). If you find any of these things to be untrue, then please let me know so I can correct them.

Now let's begin.

 

Hard hand-holding tutorials

A slick way to SSH into your AWS EC2 instance [Ubuntu/Cygwin] (July 2011)

How to Set Up Dedicated MySQL Databse Server Using Amazon EC2 (July 2011)

Tutorial: Setting Up Apache/SSL to Serve HTTPS (July 2011)

Python Web-page Scraping - Installing lxml and Beautiful Soup (June 2011)

How to automate slugs (and other fields) in Django Models (June 2011)

 

Read-between-the-lines tutorials

Generating Python documentation from comments using Sphinx, Epydocs, or Pydocs (October 2011)

Get client IP addresses using Nginx and Django (July 2011)

Posting code snippets on blogs, use Github/Gist (June 2011)

 

Wesley-approved external tutorials

"Write failed: Broken pipe" fix for AWS ec2 SSH (September 2011)

Fix: Python MySQLdb Insert Statements Not Saving (September 2011)

Dealing with the hassles of 'syncdb' in Django - how to use South (September 2011)

Facebook OAuth in PHP and PHP with Nginx (September 2011)

Getting Python to work with Facebook's OAuth 2.0 auth system (August 2011)

Deploying Django on Nginx (July 2011)

Nginx configuration solutions for Comodo SSL and serving static files (July 2010)

Generating Python Documentation from comments using Sphinx or Epydoc or Pydocs [UPDATED]

So I recently created a Python wrapper for the Yipit API and wanted to create some legit looking docs. To do so, I found some free things to help out from StackOverflow.

The things I ended up trying out were Sphinx (beautiful, but the quick generation is not super detailed), and Epydoc (not as beautiful, but very detailed and well-structured...so my favorite out of the two).

The Sphinx tutorial on the site is slightly confusing (or at least it was to me) so I used this better Sphinx tutorial. I followed it to the 't' and it worked out. Actually it is a little old so there are TWO things that are slightly different but not a big deal.

The Epydoc tutorial on their site is VERY easy to follow. It's super easy and this is basically all you have to do (after installing - which I did from source - of course).

 $ epydoc --html the_name_of_your_module -o the_folder_you_want_the_docs_in 

Boom. Simple as that. Just comment your code super well, and it will be AWSome :). 

************
UPDATE
************

Just realized python has something built in that lets you generate just one html file that is pretty sufficient (and I would say potentially even better than Sphinx's quick generation).

Here's how


$ pydoc -w package_name

And that should generate a file in the directory you called it called 'package_name.html''. You just need to make sure that package_name is in your PATH.

More Nginx configuration solutions

I'm mostly putting these up here so that I can remember them...

But for those of you who are getting your (free) SSL certificates from Comodo, I found this link that tells you how/what to concatenate your .crt files into for Nginx: http://forum.nginx.org/read.php?11,125477

And here is a server configuration file for Nginx that I was able to use to figure out how to properly serve my static files: https://code.djangoproject.com/wiki/ServerArrangements

Just got our site working with Nginx (was using Apache)!

Thanks a lot to the tutorial here:

http://www.dmclaughlin.com/2008/11/03/complete-guide-to-deploying-django-on-ubuntu-with-nginx-fastcgi-and-mysql/

We were currently running Apache (which btw took years to configure...) and I just spun up an EC2 instance and got our site deployed on it using Nginx super quickly! Will maybe write more details on this later but that tutorial really guided me step-by-step and worked perfectly...

Now I have to go through my old SSL tutorial and set that up. Overall though...feelin like a baus.

Tutorial: A Slick Way to SSH into your AWS EC2 Instance [Ubuntu/Cygwin]

So I found out there are some tricks to both getting SSH into your ec2 instance to work, as well as getting it to work in a slick manner. My goal is for you to be able to simply type:

aws_ssh

into your terminal, and for that to magically work! Only the first two steps are really relevant in getting it all working, but the next few make it slick and AWSome...

Here's what I'm going to show you:

  1. Set up proper permissions on your .pem file
  2. Test SSH in Terminal
  3. Make an executable file to SSH
  4. Create a shortcut to that SSH

SET UP PROPER PERMISSIONS ON YOUR .PEM FILE

So the way AWS sets it up is that if you try to SSH into your instance with a badly permissioned key (.pem) file then it will send some message like "ZOMG we need your key to be more secure, permission denied, public key" (btw that was totally paraphrased). But anyways, what you need to do is simply change permissions on the file to something a little more secure (like owner read-only) and AWS will be a little less temperamental.

sudo chmod 600 /path/to/key.pem

BOOM. Now that should be it for that.

 

TEST SSH IN TERMINAL

This is just a basic test to make sure that your .pem file is up-to-par now and everything is working. So go ahead and try this out:

ssh -i /path/to/key.pem ubuntu@your.aws.public.dns

This should work! If it does sir/ma'am then proceed. If not, it may be one of a few things.

  • Double-check your ec2 public dns is right, for some reason Amazon has changed mine up once or twice...
  • Make sure you typed ubuntu@ in front of your DNS if you're using an ubuntu instance. I believe that a CentOS instance uses root@, and I have no idea for the others...

 

MAKE AN EXECUTABLE FILE TO SSH

So now that you've made sure SSH'ing into your ec2 instance actually works, let's start doing some magic. The first thing you'll have to do is create the executable file and then make it an executable file.

Create the executable file

Create that file somewhere (I'm calling mine aws_ssh.sh. Name yours w/e but just make sure it ends with .sh) and make sure you add the hash-bang so it knows to use bash and also include the ssh -i line that worked in the last step.

Make it an executable file

 To make this executable it's pretty easy... just do the following inside the directory that holds aws_ssh.sh.

chmod u+x aws_ssh.sh

Now you should be ready to go with the executable file! To double check that the file works just type this in Terminal:

/full/path/to/aws_ssh.sh

 

MAGICALLY CREATE THE SHORTCUT TO SSH

This next part is magical (for noobs like myself)! We are going to create a magical short-cut for you to easily ssh into your instance. Now that you know how to ssh from Terminal, and can do it all through an executable file (so you never have to memorize the DNS), we're going to create a simple one-worded shortcut to launch that executable from your Terminal!!

Essentially we're just going to create an alias from a keyword to that executable file (woops, did I just ruin the magic part?).

Here's how we do it:

Open ~/.bashrc and add the following line to the bottom of the file somewhere:

Just replace aws_ssh with whatever keyword you'd like to assign and you should be good! Just make sure you don't add any extra spaces or it won't work. To test this out EXIT TERMINAL, re-open it and test out your keyword. Type aws_ssh (or w/e you chose) and hit enter and you should have success!

 

As always if you guys run into any issues feel free to http://wesleyzhao.com/pages/contact me!

Tutorial: How to Set Up Dedicated MySQL Database Server Using Amazon EC2

DISCLAIMER: This is my current setup. Users with other distros beware.

So in the spirit of learning about scalability (and also after running into some issues dealing with several databases running on different local development environments we are using) I decided it would be a good idea to set up an EC2 instance solely as a database server. This tutorial will use MySQL as the database of choice not necessarily because I think it's the best, but because it is what I'm used to (I know... I'm lazy). So here's what we need to do.

  1. Assume you have an EC2 instance running for this
  2. Install mysql, apache2, php5, and phpmyadmin
  3. Configure your MySQL
  4. Configure your EC2 instance

ASSUME YOU HAVE AN EC2 INSTANCE RUNNING FOR THIS

Done. 

(JK, kinda. Here's a tutorial for this. Also I currently use the AMI mentioned above in the disclaimer. BTW You will need to SSH into your EC2 instance. Amazon has a pretty good tutorial if you just right click your instance in the management console, and click Connect).

INSTALL MYSQL, APACHE2, PHP5, PHPMYADMIN

Alright, now assuming we are running a clean ec2 instance, here are some of the dependencies you'll need. But before you do anything, just make sure your packages are up to date with the following code in your terminal.

sudo apt-get update

Install Apache (ignore this if you don't plan on using phpmyadmin or another front-facing MySQL app)

sudo apt-get install apache2

That was easy.

Install MySQL

 

sudo apt-get install mysql-server

During the install it might ask you if you want to preconfigure this for Apache. I almost got confused here, but click the Space Button to select, then Tab to hit "OK."

Oh, also, make sure you fill in a password for root (the more secure the better...but write it down).

Install PHP (ignore if you don't need phpmyadmin)

sudo apt-get install php5 sudo apt-get install php5-mysql sudo apt-get install libapache2-mod-php5

To double check everything is working by going into /var/www and adding a php file and try loading it. You may need to restart the apache2 server to check this by doing:

sudo service apache2 restart

Install PhpMyAdmin (not required if you don't think you'll need this)

sudo apt-get install phpmyadmin

Boom! So far so good....

Now try going to the webpage by typing: http://your.ec2.public.dns/phpmyadmin

Cool if it worked. If not bummer :(. Sorry, try again? 

 

CONFIGURE YOUR MYSQL

These next parts are not for the weak....also on a more serious note, these instructions may be specific to my Meerkat Ubuntu server.

But ah yes, what you'll need to do to configure your MySQL is two-fold.

Allow access from an outside IP

You'll need to navigate to your my.cnf file by typing:

sudo emacs /etc/mysql/my.cnf

Then change bind-address from 127.0.0.1 to 0.0.0.0. For me this was on line 52 and should look like this.

bind-address                 = 0.0.0.0

Grant MySQL user permission access from outside IP

Note: replace root with the user you chose (default is root). Replace securepassword with your password. You can change % if you want, as it is, it will allow access from any ip address.

Now you're at the final step my friend...

 

CONFIGURE YOUR EC2 INSTANCE

All you'll really need to do here is edit the security group of your instance to allow for the MySQL default pport (3306), and the HTTP port (80, if you plan to use phpmyadmin or something like that). 

To do this:

>go to the AWS Management Console

>Navigate to your EC2 Instance, and locate the name under the "Security Group" column

>Click Security Groups on the left bar, and then click the name of the security group assigned to your instance

>Click Inbound on the bottom

>Under Create a rule use the dropdown menu to find MYSQL and HTTP and add those both one at a time

>Make sure to click Apply Rule Changes or it won't save.

 

YEEE. Now you shall be ready to go. This will allow you to have a dedicated EC2 instance for all your MySQL database requests. 

Here are some of the tutorials I used to get this info: installing it all and configuring MySQL.

Tutorial: Setting Up Apache/SSL to Serve HTTPS

DISCLAIMER: This is my current setup. Users with other distros beware.

I've just successfully figured out how to build trust in my website for FREE (during my 30 day trial)! I wanted to record it down for myself and hopefully relieve some of the frustration from others who may be trying to figure it out, but have no consolidated tutorial. Again - note the DISCLAIMER above. Along with that fact, this tutorial is basic and will only set it up so all pages are served via HTTPS - will figure out how to do the mix later...

Here are the basic steps we'll walk through to get this to work:

  1. Get an SSL certificate
  2. Install your SSL certificate
  3. Configure Apache to serve as HTTPS
  4. Hope for a miracle

Now... let's do this.

GET AN SSL CERTIFICATE

The first thing you'll have to do is choose where you want to get it from. I was directed to PositiveSSL from a trusted ex-Googler/friend and so that's what I went with. More specifically, because I was just playing around I decided to go with their free trial before I shelled out some ca$h-money. Here's where you'd go for that

PositiveSSL - Free SSL Certificate for 30 days

So now that we are at the page, you need to click through, then discover you'll need to copy/paste a Certificate Signing Request (CSR).

Get a CSR

I would personally make a folder to termporarily store this stuff:

mkdir ssl_stuff
cd ssl_stuff

Then type the following to generate both your private key (the .key file) and public CSR (the .csr file):

openssl req -nodes -newkey rsa:2048 -keyout myserver.key -out server.csr

It will then prompt you for some fields and most of them are optional...but why not:

Enter in your domain name in replacement of mydomain.com under 'Common Name.' It is also suggested that you make a backup of your myserver.key (or private key) just in case (I made mine myserver_backup.key).

Meow, copy paste the contents of your server.csr file back into the webpage and continue on to get your free certificate!

NOTE: You'll have to fill out a couple more annoying forms and hit complete!

Finally, you'll have to wait to get about 3 emails, the first will just be some rando confirmation, then a authentication email, then finally you'll get the email with the zip file of your SSL certificate!!!!!! Onward.

*Where I got my info : Comodo Support CSR Generation

 

INSTALL YOUR SSL CERTIFICATE

Boom! be proud you made it this far. First thing you'll have to do is unzip your SSL certificate, and use Filezilla (or your favorite FTP client) or FTP straight up from Terminal like a boss to move those files (preferably) into the same folder you kept that other ish in. So hopefully the following command will work inside the ssl_stuff folder:

ls   mydomain_com.crt mydomain_com.ca-bundle myserver.key  myserver_backup.key server.csr

Now you'll have to move the private key and certificates to the correct Apache folder. Depending on your distro it will still probably be in /etc/ssl/ somewhere under private and certs folders but here is what I had to do:

KK COOL. Now you're going to have to configure Apache to find and use the SSL Certificate.

You'll need to find where you have your VirtualHost stuff set up but mine was in /etc/apache2/sites-enabled/000-default. Here's what you'll have to add to the file somewhere inside the VirtualHost tags:

*Where I got my info: Comodo Support Ceritificate Installation

Now run the following to just confirm that it works.

sudo a2enmod ssl #enable SSLEngine etc. to work
sudo service apache2 restart

If everything works fine and dandy you're good! If no, it's probably a spelling error...shame on you. 

 

CONFIGURE APACHE TO SERVER AS HTTPS

Alrighty troll, we're almost there. We've got most of this set up. Now we just have a few last Apache configuration steps so stay with me.

Tell Apache to redirect all HTTP requests to HTTPS

Add the following to your Apache httpd.conf file:

And type the following in terminal to enable RewriteEngine (etc) to work and then to check for spelling errors:

sudo a2enmod rewrite
sudo service apache2 restart

Tell Apache to serve through the HTTPS port

The default port being used for your pages is probably 80, which you need to switch to 443. It's as simple as just replacing 80 with 443 in your 000-default file. So it should look something like this:

Open up the HTTPS (443) port on your instance

So navigate over to your AWS Management Console and click:

  • EC2 (on top toolbar)
  • Instances (left toolbar) >> then read what security group it is
  • Security Groups (left toolbar)
  • Name of Security Group your instance was assigned
  • Inbound (new loaded panel on bottom)

And now under 'Create new rule' selected 'HTTPS' and click Add Rule. Finally click Apply Rule Changes (DON'T forget this).

 

HOPE FOR A MIRACLE

Ah young padawan, your final task:

sudo service apache2 restart

YAY! Now if you try to navigate to any page, it should automagically load as an HTTPS url rather than HTTP and everything should work! If not then... well... let me know! I will definitely be much more helpful if your setup is the same as mine, but if not I will do my best!

 

FINAL_NOTE_1: If Chrome/whatever-browser is saying you have unsecure content but loads your SSL certificate, it is probably because some of the scripts (like JQuery or Webfont) you are loading are via HTTP request. Simply change those to HTTPS, re-open your browser, and it should all work!

FINAL_NOTE_2: Here are some other sites I used while making this:

How to automate slugs (and other fields) in Django Models

Going to post this really quick because I have to keep coding..

Here is something I learned really quick and for my sake I want to record this down, and I thought it might be usefull to ya'll as well.

If you want to automate slug creation for your Model here is how you would do it:

That is how you do basic slugify-ing. Basically you have to make sure you define the slug field, but set a default so nothing is required of you when you first instantiate the Model. Then override the Model's default save method to slugify your title before saving. I'm still trying to figure out how to make a fail-safe so that all slugs are unique and nothing clashes...

Here is the offiical django documentaion on how to override saving.

Writing my first Python/Django app - already proved I'm fallible

Sorry guys (the one or two of you, or just me, that still reads my blog) for being incognito these past couple weeks. Finals hit then got home and enjoyed myself a bit with friends and my girl. I've been working on getting up to speed on learning a python web framework for a while and I chose Django because I heard there was a lot of support out there and it seems to be taking the lead because it is both high-level enough to do cool things easily and editable enough that it gives you low-level access (unlike Ruby?). So I've been stuck on this hiccup for setting up the Django environment which (I later found out, thanks to my Dad) involved Cygwin using a unix file system and my Python still only recognizing Windows file system stuff. I finally figured it out after slaving hours and hours scouring the web for answers and asking my dad for help (he actually ended up patching it up temporarily). I was so happy that I could move on to the actual learning of Django vs the set up. Then I left it, got home after a bit, and started using my home desktop (I had set up Django on my laptop previously). I was about to write a tutorial for setting up Django on Windows when I got curious about... could I just have set it up super easily, and avoiding that whole filesystem incompatibility nonsense if I just used the Windows command prompt to set up Django... So I started just using Windows command prompt and it worked (so far)! WOW. Wasted time slaving over trying to figure it out using Cygwin on my laptop...lesson learned though. Avoid tunnel vision. I will continue my Django tutorial on both my laptop and my desktop (I figure it will be helpful anyways to do it twice...and engrain it into my mind) and if both work out fine then I will simply write my tutorial on how to set up Django with windows using Windows command prompt and avoiding the mess I fought with!! Unless someone requests for me to write the version for using Cygwin as well...would be more than happy to. Anyways...goodluck to me on getting this all working. And as soon as I figure out if both work/not I will write that tutorial!