A Python Yipit API Wrapper - my first real contribution to open source

So I've been playing around with this idea to mashup a daily deals API with Yelp to get emails only for deals at good restaurants and I knew I wanted to use Yipit. I looked at their API and realized they didn't have libraries/wrapper for anything and so I decided to create one for Python.

I figured a) it would be useful to me to use in the project, b) it would be useful to other python guys using Yipit, and c) it would be useful to me to just do it and learn on the way.

So I finished it and here it is: https://github.com/wesleyzhao/python-yipit-wrapper

AW YEAHHH.

I was able to even do some cool abstract class stuff with Python, learn how the whole **kwargs argument works in python, generated two awesome sets of documentation, and some other stuff. I structured it very similarly to the python-twitter wrapper, so thanks to those guys.

You can easily install the python-yipit wrapper by downloading the tar, unzipping it and...

 $ cd python-yipit-wrapper 
$ python setup.py install 
$  >>import yipit  
 

Generating Python Documentation from comments using Sphinx or Epydoc or Pydocs [UPDATED]

So I recently created a Python wrapper for the Yipit API and wanted to create some legit looking docs. To do so, I found some free things to help out from StackOverflow.

The things I ended up trying out were Sphinx (beautiful, but the quick generation is not super detailed), and Epydoc (not as beautiful, but very detailed and well-structured...so my favorite out of the two).

The Sphinx tutorial on the site is slightly confusing (or at least it was to me) so I used this better Sphinx tutorial. I followed it to the 't' and it worked out. Actually it is a little old so there are TWO things that are slightly different but not a big deal.

The Epydoc tutorial on their site is VERY easy to follow. It's super easy and this is basically all you have to do (after installing - which I did from source - of course).

 $ epydoc --html the_name_of_your_module -o the_folder_you_want_the_docs_in 

Boom. Simple as that. Just comment your code super well, and it will be AWSome :). 

************
UPDATE
************

Just realized python has something built in that lets you generate just one html file that is pretty sufficient (and I would say potentially even better than Sphinx's quick generation).

Here's how


$ pydoc -w package_name

And that should generate a file in the directory you called it called 'package_name.html''. You just need to make sure that package_name is in your PATH.

Solved: Python MySQLdb Insert Statements Not Saving

So it turns out that if you are using MyISAM as your MySQL engine, you just need to do a simple:

conn = MySQLdb.connect("""CONNECT ARGS""")
cursor.execute(""" INSERT COMMANDS """)

And everything will be OK.

If, however, you decide to use InnoDB (which you probably should) or if for some reason you find that your INSERT commands are not saving you need to do the following...

conn = MySQLdb.connect("""CONNECT ARGS""")
cursor.execute(""" INSERT COMMANDS """)
conn.commit()
 

Here's a good explanation of how it all works if you just can't seem to be getting it right:

http://www.kitebird.com/articles/pydbapi.html

Getting Python to work with Facebook's OAuth 2.0 authentication system

I recently treaded into the dreaded territory of the Facebook API to try to create a viral app on top of AvantCard. I ran into a lot of snags that I'll share, and luckily found the solutions (which I will share as well).

The big thing I learned the hard way was that Facebook is requiring OAuth 2.0 authentication by September 1, 2011 and their Python-SDK does not support that.

I found this out by first not trying to use OAuth 2.0 and being OK with the Python-SDK, but discovered that the non-OAuth Facebook cookie (with the key "fbs_[your_app_id]") was not storing in Chrome for some reason. After I figured that out, I switched to OAuth 2.0 by changing a setting within FB.init() in their Javascript SDK and was able to get the correct cookie saved (with the key "fbsr_[your_app_id") in chrome! But... even with the cookie, the Python-SDK wasn't getting it. 

So I went into facebook.py and tweaked the cookie-getter function to look for "fbsr_" vs "fbs" and then discovered they were just completely different and even doing that wouldn't fix the problem.

And along came this handy-dandy website that clarifies all this. They pointed out the problem I discovered after trying to deal with it myself, and created a workaround. Generate the login-url, and authentication-url's with Python. Don't let Facebook's Javascript SDK suck you in.

So I did just that, and it allowed me to successfully, get a user to the log-in screen, and generate the cookie, and get the cookie to get access to the Facebook access_token for that user. The only thing I need to do is throw in some JQuery to let the user not have to physically switch web-pages to do a Facebook login, and simply have a pop-up.

 

The other handy-dandy thing I fell upon was a pre-made function to handle FQL queries in Python really easily by just adding some code into the facebook.py file in the Python-SDK. Here's the site. Even though the site has you put the function inside facebook.py, I just modified it slightly to work in my controllers file for my Django/Python app.

 

Hopefully if anyone else out there is trying to get their Python app to work with Facebook's new OAuth 2.0 authentication system, you found this helpful.

My Amazon EC2 Instance

I've used a lot of tutorials on the web regarding installing/configuring/herp-derping my ec2 instance for my projects and have one thing to be consistent: there is no consistency...

So I've put it upon myself to try to be as consistent (or at least fully transparent) on the configuration/system I am running so that someone who happens to be using my exact setup can follow it step by step, and someone who is not can know right away that the reason my tutorials don't work might just be because of a difference in setups.

So let's get the details out right now:

aws image id - ami-cc405a5 [probably the most important thing to know]

os - Ubuntu Maverick Meerkat [tried upgrading to Natty, f'd everything up]

python - 2.6.6

HTTP server - Apache2

There is probably several more things I need to specify, but for now this is good. Please let me know if you want me to specify anything else, and I will add as I think of important things.

Python Web-page Scraping - Installing lxml and Beautiful Soup

So I've always used RegEx to scrape all my data. In fact, it can get pretty tough/tedious for a noob like me. I've been able to use it, but it's just a hassle. And until a few days ago, I thought this was my only route.

Fortunately for me, a few super-smart-engineer-entrepreneur friends (Noah Ready-Campbell and Calvin Young) told me about lxml and Beautiful Soup. They said it was a little tricky to install, but I didn't believe them... I tried it out for myself and actually had a lot of trouble getting it going. Eventually I stumbled upon something that made it pretty easy for me, but I'm hoping to turn that around and make it even easier for you to get.

So here it goes (disclosure: this worked on my Ubuntu EC2 emi and Ubuntu home machine):

How To Install lxml:

UPDATE: 

Try:

sudo apt-get install python-lxml python-beautifulsoup

Thanks to lamby of HN for this! I just tried it and it worked on my new Ubuntu EC2 ami...if anyone finds out this doesn't work please report it to me/someone-actually-important!

/UPDATE

The problem people usually have is there are just a lot of dependencies and it just seems that it never works. So here is what we'll end up getting through this:

  • libxml2 - the lxml library
  • libxslt1.1 - some other library that is a dependency
  • libxml2-dev - the libxml dev header
  • libxslt1-dev - the libxslt dev header
  • python-libxml2 - python bindings for libxml2
  • python-libxslt1 - python bindings for libxslt1
  • python-dev - python dev headers
  • python-setuptools - the thing that lets you run easy_install

So here's how it should all look like:

And if some of these exact commands don't work try searching for the package or updating your package directory:

 

Boom! Should be done. If you guys are running Ubuntu and have issues with this feel free to email me: wesley.zhao@gmail.com

 

How To Install BeautifulSoup:

This is much easier and hardly needs any instruction.

First go here to find the file you want: Beautiful Soup Downloads

Then depending on your file path and download you choose this is how you do it:

 

Once you have these installed check this post out: lxml; an underappreciated web scraping library.

The post has some great examples of how easy it is to scrape with lxml and BeautifulSoup. It's practically like being able to grab CSS tags!

Again if anyone has any questions feel free to email me! I know the set-up process can be a huge pain so...yeah.

How to automate slugs (and other fields) in Django Models

Going to post this really quick because I have to keep coding..

Here is something I learned really quick and for my sake I want to record this down, and I thought it might be usefull to ya'll as well.

If you want to automate slug creation for your Model here is how you would do it:

That is how you do basic slugify-ing. Basically you have to make sure you define the slug field, but set a default so nothing is required of you when you first instantiate the Model. Then override the Model's default save method to slugify your title before saving. I'm still trying to figure out how to make a fail-safe so that all slugs are unique and nothing clashes...

Here is the offiical django documentaion on how to override saving.

Writing my first Python/Django app - already proved I'm fallible

Sorry guys (the one or two of you, or just me, that still reads my blog) for being incognito these past couple weeks. Finals hit then got home and enjoyed myself a bit with friends and my girl. I've been working on getting up to speed on learning a python web framework for a while and I chose Django because I heard there was a lot of support out there and it seems to be taking the lead because it is both high-level enough to do cool things easily and editable enough that it gives you low-level access (unlike Ruby?). So I've been stuck on this hiccup for setting up the Django environment which (I later found out, thanks to my Dad) involved Cygwin using a unix file system and my Python still only recognizing Windows file system stuff. I finally figured it out after slaving hours and hours scouring the web for answers and asking my dad for help (he actually ended up patching it up temporarily). I was so happy that I could move on to the actual learning of Django vs the set up. Then I left it, got home after a bit, and started using my home desktop (I had set up Django on my laptop previously). I was about to write a tutorial for setting up Django on Windows when I got curious about... could I just have set it up super easily, and avoiding that whole filesystem incompatibility nonsense if I just used the Windows command prompt to set up Django... So I started just using Windows command prompt and it worked (so far)! WOW. Wasted time slaving over trying to figure it out using Cygwin on my laptop...lesson learned though. Avoid tunnel vision. I will continue my Django tutorial on both my laptop and my desktop (I figure it will be helpful anyways to do it twice...and engrain it into my mind) and if both work out fine then I will simply write my tutorial on how to set up Django with windows using Windows command prompt and avoiding the mess I fought with!! Unless someone requests for me to write the version for using Cygwin as well...would be more than happy to. Anyways...goodluck to me on getting this all working. And as soon as I figure out if both work/not I will write that tutorial!

Python for web - the next frontier (for me)

I just had a great conversation with Rob Eroh, VP of Product Development of Milo, over the phone tonight. He had some great insight on how Milo started, the work they put in, and the things entrepreneurs should think about when starting a consumer web start up. Along with being a great guy, Rob and I talked about the type of codebase that was behind Milo and the expertise I had. I am coding in mainly PHP (I used some Python in my VERY first project), while Milo is build mainly on Python. Rob has directed me to a Python framework called Flask, which he likes a lot. I have decided that maybe it is time for me to learn how to build both more scalable code, and code in something other than PHP. So, I have promised myself my next new project will be written in Python on top of the Flask framework. I am hoping there is not too much of a learning curve and there won't be too many (more than 3 lets say) just staying up learning it all. But, it will be a worthwhile experience that I am sure will be useful for me down the line. Also...if Rob is reading this, don't worry I did not forget to look into Hudson.