elblogg

BrightKite is a service, much like twitter actually, but that uses your location to find interesting stuff. It could be a great tool to create ad-hoc gatherings of some group of people, for instance coders. If there were a lot of BrightKite users in Bergen for instance, I could leave a message on brightkite that I and some other Python coders were going for a beer at Henriks, and that any others that is interested could join if they wanted, together with the GPS coordinates for Henriks. Other BK users in Bergen interested in Python could then see that message based on the proximity to that location and maybe the search term BrightKite.

The problem is that, though there are BK applications for the iPhone, theres no client for the S60 mobile market, and also the iPhone app provides a poor precision on the location.

Unofficial Google Latitude T-Shirt by Matt Jones ( moleitau on flickr)

This post is going to tell you how to use the data provided by this service in your Python apps. Upcoming posts will tell you how to turn this data into an address, using the Google Maps API, and the final post will show you how to use this address to update your BrightKite location through their API.

I strongly discourage using this badge as it is, as it exposes your private URL’s for your location information, and once these are available on another page than the google latitude badge page, you can never use the the webservices I’m about to tell you about, without possibly having strangers get to it.

As long as you keep these URL’s private, you should be pretty safe.

Go to the Google Latitude Badge page, and enable the service at best available precision, then go to the bottom of the page, where you would find a public json feed. It’s url looks someting like this: http://www.google.com/latitude/apps/badge/api?user=YOUR_SECRET_ID&type=json We’re going to use that URL in a moment.

The script below defines a GLatitude class, which we will initialize with your secret id (the long number from the json feed url), and which we then can use to get the latitude and longitude of the current location the Latitude service has on you.

Note that you can specify your location yourself using the privacy settings in the Google Maps application, so you can perfectly well lie about your location.

import urllib2, json
class GLatitude:
  def __init__(self,uid):
    self.uid=uid
  def getLocationData(self):
    return json.load(urllib2.urlopen("http://www.google.com/latitude/apps/badge/api?user=%s&type=json" % self.uid))
  def getCoords(self):
    return self.getLocationData()['features'][0]['geometry']['coordinates']

It’s really simple. All I do is to fetch the json data, and lets the json module parse it. then the getCoords() method extracts the coordinates from the data returned.

I placed this code in a file called GoogleMaps.py, which we also will use in the next post, so I can use it as a module later on.
To use this then, you could do the following:

import GoogleMaps
gl=GoogleMaps.GLatitude("YOUR_SECRET_ID")
myLocation=gl.getCoords()
print "I'm located at latitude %s and longitude %s " % myLocation

Running this would produce something like the following:

I’m located at latitude 60.390218 and longitude 5.330515

Stay tuned for the rest of this tutorial.

During the last week there has been a couple of great releases in the software I use every day.

First out was Python 3.1. It was released June 27th and features

• An ordered dictionary type
• Various optimizations to the int type
• New unittest features including test skipping and new assert methods.
• A much faster io module
• Tile support for Tkinter
• A pure Python reference implementation of the import statement
• New syntax for nested with statements

Next out was Mozilla Firefox 3.5, which was released earlier today (June 30th), most noteworty items from that release is the @font-face css-attribute, that allows web-designers to bundle custom fonts with their website designs, and use custom fonts in their design.  I have used that feature to change the header of my blog, you would se a grudge font if you use the new Firefox or Safari. They have also released a new javascript-engine, which they claim to be a lot faster than the old one. Also, support for the <video>-tag has been added.

Then last, but not least, the long awaited PHP 5.3.0 has been released, which provides namespaces and closures for PHP.

Lets say you have two lists, one of them containing the values [1,2,3] and the other one containing the values [4,5,6], and you want to combine these two lists into one list, like this one: [1,2,3,4,5,6].

Let’s see how this can be done in different (high level) programming (or scripting or templating) languages.

Java

Java has multiple list implementations, lets start with the most basic one, the builtin array.

int[] A = {1,2,3};
int[] B = {4,5,6};
int[] C = new int[A.length + B.length];
int counter=0;
for(int e : A){
  C[counter]=e;
  counter++;
}
for(int e : B){
  C[counter]=e;
  counter++;
}

or with an ArrayList:

/*Dropped importing the correct classes, 
and constructing the ArrayLists A and B, since thats not the focus here...*/
List<Integer> C = new ArrayList();
C.addAll(A);
C.addAll(B);

PHP

So, how is the same done in PHP?

$A=array(1,2,3);
$B=array(4,5,6);
$C=array_merge($A,$B);

Python

And at last, how does the Pythonistas do this?

A=[1,2,3]
B=[4,5,6]
C=A + B

Note: the rubyists use the same approach.

I just wrote a description on how you can use apache to create an authenticated proxy ahead of your development server, or other http-based servers that doesn’t provide authentication themselves.
You can find it under the docs-section or by clicking here: Using apache to create an authenticated proxy.

The chumby device is an interesting device, being one of the few open source electronic gadgets and all. But sadly, its the hardware that’s open source. And for me that is really a bummer, since I’m a software hacker. I’m not that confident with the soldering iron. That means that the fact that the hardware in this device is open source doesn’t matter that much to me, especially since I’ve found only one producer and reseller of the Chumby, namely the chumby store. And they don’t even ship to Norway!

And, the stuff that I find interesting with such a device is the ability to make my own software for it, displaying the stuff I want it to, and controlling the other stuff in my home from it. (Like the HTPC system etc.). And thats where the chumby really disappoints me. First off, I’d need to program the user interface in Adobe Flash, which is a thing I don’t know how to do, and I don’t have the tools. The open source support for creating flash applications are appalling too, so to make good applications I would have to buy the commercial non-open software from Adobe. Which don’t even run on Linux. So then I’d have to install Windows on my machine to run it etc. etc. I don’t hate my freedom that much.

On the other hand, I could get an Nokia N810 device, the hardware there is closed source, but the software is completely open. I could program my applications using Python and GTK. Even more, the N810 is more powerful, it has a bigger display, and the device is smaller. Okay. It’s not smaller than the actual electronics inside the chumby, but its a lot smaller, or at least thinner than a unmodified chumby. It even can run from batteries. And, there’s nothing that prevents me from doing the same type of hacks that people does with the chumby. Very few people actually modifies their chumby, they mostly modify the casing of the chumby. Like this guy:

(Chumby hack published in MAKE Vol. 16)

Okay, this post turned out to be a rant, but live with it.

If you’re really trying, you will always be able to write completely unreadable code in any programming language. All the language designers attemts to force you to write readable code, and even making it so the easiest way ahead, even for quick hacks is to write it as readable and reusable code will fail if you’re really trying to write unreadable code.

This guy is obviously concerned about “job security”. (not the blog author, but the code author)

#!/usr/bin/env python
import os,sys
C=os.chdir
S=os.system
M=os.mkdir
J=os.path.join
A=os.path.abspath
D=os.path.dirname
E=os.path.exists
W=sys.stdout.write
V=sys.argv
X=sys.exit
ERR=lambda m:W(m+"\n")
PRNT=lambda m:W(m+"\n")
assert len(V)==2,"you must provide a sandbox name"
SB=V[1]
H=A(D(__file__))
SBD=J(D(H),SB)
C(SBD)
PST=J(SBD,'bin/paster')
VAR=J(SBD,'var')
ETC=J(SBD,'etc')
S("mkdir -p "+VAR)
PRNT("restarting "+SB)
CMD=";".join(['source %s'%J(SBD,'bin/activate'),PST+" serve --daemon --pid-file=%s/sandbox.pid --log-file=%s/sandbox.log %s/sandbox.ini start"%(VAR,VAR,ETC)])
PRNT(CMD)
S(CMD)
PRNT("All done!")
X(0)

Aaannd… Heres my understanding of the code:

#!/usr/bin/env python
import os,sys
 
#bail out if there's no sandbox specified
assert len(sys.argv)==2,"you must provide a sandbox name"
sandbox=sys.argv[1]
 
#get the dir of this script
home=os.path.abspath(os.path.dirname(__file__))
 
#sandboxdir is this dir/sandboxname
sandboxdir=os.path.join(os.path.dirname(home),sandox)
 
#go into the root of the sandbox
os.chdir(sandboxdir)
PST=os.path.join(sandboxdir,'bin','paster')
VAR=os.path.join(sandboxdir,'var')
ETC=os.path.join(sandboxdir,'etc')
 
#make var directory if it doesnt exists
if not os.path.exists(VAR):
    os.mkdir(VAR)
 
print "restarting "+sandbox
 
# Build the system commands required to restart the sandbox
cmds=[]
cmds.append('source %s' % os.path.join(sandboxdir,'bin','activate'))
cmds.append(PST+" serve --daemon --pid-file=%s/sandbox.pid"
            "--log-file=%s/sandbox.log %s/sandbox.ini start" % (VAR,VAR,ETC))
CMD=";".join(cmds)
PRNT(CMD)
 
#run the cmds
if os.system(CMD) == 0:
    print "All done!"
    sys.exit(0)
else:
    print "Ooops. failed."
    sys.exit(1)

… is greater than zero.

Python is a dynamically, strong typed programming language. In my opinion that makes it more important to have good tools than for statically typed languages. Sadly, it also makes it more difficult to create great code analysis tools and IDEs. Meanwhile, I find more people in the Python community that is happy coding with Vi(m) even for quite large projects, than in the Java community. My guess is that that is because the tool support for Python has been poor.

When your IDE shouts at you with red lines under your function calls because it’s not able to look up in your custom libraries, of course you’ll refuse to accept it, and go back to your simpler editor.

Torbjørn Nordbye implemented the awesome Ruby support for Netbeans, and has now started on the work to give us Python awesomeness in Netbeans as well.

As mentioned in the comments of a previous blog post I have a good example of simple, obvious bugs that could stay in your application for years, without anyone noticing them, because that piece of code run almost never. Bugs that good tools, and/or compilation of staticly typed languages would find for you, but that unittesting and manual testing of your application probably wouldn’t find.

This is a bug that Tor found while testing his Python support for Netbeans, when he pulled in the datetime module that ships with Jython. There, in some exception handling the code references the non-existent class ValuError, which probably should be ValueError. A simple typo that has been undiscovered for a long time.

We all make errors. Good tools make us do less of them, and find them faster.

Okay. It’s kind of stupid to put static typing up against Unit tests. Even so, lots of people does so a lot of times in the discussion between dynamic and static programming languages.

Static typing provides you with a security net when it comes to typos. Alhough, it doesn’t give you any security against logical errors.

Read the rest of this entry »

I spotted this meme on Steve Holden’s blog.

• Grab the nearest book.
• Open it to page 56.
• Find the fifth sentence.
• Post the text of the sentence in your journal along with these instructions.
• Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

So. In my case, the closest book to me when writing this post was Python phrasebook by Brad Dayley.

The fifth sentence on page 56 is as follows:

Defining a list in Python is a simple matter of assigning a number of Python objects to a variable name using the = operator.

So now you know.

So, you’re going to parse a webpage, to extract some information. For instance if you want to get the tracking information for your last online order, and you want to display the tracking information changes using growl, dbus notifications or xosd.
You know regular expressions, so you go to the job with your long range missiles ready. But wait a minute, you’ll probably solve the problem but is regular expressions really the right tool?
The pro for regular expressions is that you can use the same tool you always use for parsing jobs, but then again you doesn’t learn anything new out of this. You might fortify your position as regex wizard even more, but how about something completely different?

Now. Most webpages is written in HTML, and some even in XHTML, for HTML documents languages like Python has a built-in parser, after the model of the SAX-parser. (It’s probably the other way around, the SAX parser is built on the base of the HTML parser…) Most programming languages has good support for XML, so for XHTML documents, you can use the HTML-parser, a SAX-parser or even the XML-DOM parsers.

The benefit of doing it this way is that your parser will probably be more robust to minor changes in the webpage. You don’t reinvent the wheel (The best way I’ve found to parse HTML documents using regular expressions is to make a specialized SAX-like parser anyway). Your code will probably be readable in a year, and others might even be able to understand your code. And finally, you learn something new, which might give you a fresh view on a lot of problems.

Now back to the original issue, to make a parser for the parcel tracking of your postal service. Here’s an example parsing the shipment tracking page of posten, the norwegian postal service.

Read the rest of this entry »