elblogg

Java Conference 2006 by Matteo Penzo ( matteopenzo on flickr)

Expression Language makes working with Java Server Pages a slightly less pain in the behind. But, its functionality is very limited, there is for instance no way to round off a number, neither as floor, ceiling or to the closest integer

Floor(foo) -> ${foo - ( foo % 1 ) }
Ceiling(foo) -> ${foo + ( 1 - (foo%1)) % 1}
Round(foo) -> ${foo +0.5 - ((foo+0.5) % 1) }

This still gives you a float, so to get an integer, you must either use the <fmt:formatNumber>-tag

<fmt:formatNumber value="${foo - ( foo % 1 ) }" var="fooAsInt" maxFractionDigits="0"/>

(really, this returns a string, representing the number without the decimals. But as EL automatically casts between strings, floats and integers, this is mostly OK.)

or you can hack it using fn:replace():

${fn:replace(foo - ( foo % 1 ),'.0','') }

(Yes, you may use singlequotes for strings in EL, even though this is supposed to be Java….)

BrightKite is a service, much like twitter actually, but that uses your location to find interesting stuff. It could be a great tool to create ad-hoc gatherings of some group of people, for instance coders. If there were a lot of BrightKite users in Bergen for instance, I could leave a message on brightkite that I and some other Python coders were going for a beer at Henriks, and that any others that is interested could join if they wanted, together with the GPS coordinates for Henriks. Other BK users in Bergen interested in Python could then see that message based on the proximity to that location and maybe the search term BrightKite.

The problem is that, though there are BK applications for the iPhone, theres no client for the S60 mobile market, and also the iPhone app provides a poor precision on the location.

Unofficial Google Latitude T-Shirt by Matt Jones ( moleitau on flickr)

This post is going to tell you how to use the data provided by this service in your Python apps. Upcoming posts will tell you how to turn this data into an address, using the Google Maps API, and the final post will show you how to use this address to update your BrightKite location through their API.

I strongly discourage using this badge as it is, as it exposes your private URL’s for your location information, and once these are available on another page than the google latitude badge page, you can never use the the webservices I’m about to tell you about, without possibly having strangers get to it.

As long as you keep these URL’s private, you should be pretty safe.

Go to the Google Latitude Badge page, and enable the service at best available precision, then go to the bottom of the page, where you would find a public json feed. It’s url looks someting like this: http://www.google.com/latitude/apps/badge/api?user=YOUR_SECRET_ID&type=json We’re going to use that URL in a moment.

The script below defines a GLatitude class, which we will initialize with your secret id (the long number from the json feed url), and which we then can use to get the latitude and longitude of the current location the Latitude service has on you.

Note that you can specify your location yourself using the privacy settings in the Google Maps application, so you can perfectly well lie about your location.

import urllib2, json
class GLatitude:
  def __init__(self,uid):
    self.uid=uid
  def getLocationData(self):
    return json.load(urllib2.urlopen("http://www.google.com/latitude/apps/badge/api?user=%s&type=json" % self.uid))
  def getCoords(self):
    return self.getLocationData()['features'][0]['geometry']['coordinates']

It’s really simple. All I do is to fetch the json data, and lets the json module parse it. then the getCoords() method extracts the coordinates from the data returned.

I placed this code in a file called GoogleMaps.py, which we also will use in the next post, so I can use it as a module later on.
To use this then, you could do the following:

import GoogleMaps
gl=GoogleMaps.GLatitude("YOUR_SECRET_ID")
myLocation=gl.getCoords()
print "I'm located at latitude %s and longitude %s " % myLocation

Running this would produce something like the following:

I’m located at latitude 60.390218 and longitude 5.330515

Stay tuned for the rest of this tutorial.

Most of the code I’ve written in my life so far has been PHP, as long as I include all the small testing of different stuff just to see if it could be done.

PHP is quick to prototype stuff in, because you quickly get something you can poke around with, there’s little boilerplate code. However, specially when I’m just hacking together something in a hurry, theres one thing that’s really, really frustrating. Specially when I deal with SimpleXMLElements.

In Python you can do this:

firstbar=element.getElementsByTagName("bar")[0]

to get the first bar element inside “element”, in Java, i can do pretty much the same, and even in javascript you could do this, but in PHP, you always have to assign the result of getElementsByTagName() to a variable before accessing it’s index.

<?
print $element->xpath("//bar")[0]; // FAILS
$bars=$element->xpath("//bar");
print $bars[0]; //WORKS
?>

Now of course, this is not the only thing that frustrates me with PHP, but this was the thing biting me yesterday.

EDIT:
I realize that the best way to do this is using the current() function in PHP:

<?
print $element->xpath("//bar")[0]; // FAILS
print current($element->xpath("//bar")); // WORKS
?>

I like using creative commons photos from flickr as illustrations on my blog, however, I find it tedious to add all the metadata to satisfy the BY directive in the creative commons license in a consistent manner. Then it’s really nice that flickr has an API where I can automatically fetch that information. I use the shortcode feature of wordpress, so I can use

[flickr url="http://www.flickr.com/photos/stuckincustoms/4099398743/in/photostream/"]

to include an image like below:

Crossing the Bridge into Old Lyon by Trey Ratcliff ( Stuck in Customs on flickr)

or

[flickr pid="4097758830"]

to insert the image below:

A View from the Ranch in Argentina and Guest Hosting on This Week in Photography by Trey Ratcliff ( Stuck in Customs on flickr)

or to get another size version i can use

[flickr pid="4088949046" version="Small"]

, like below:

The River Runs Through the Andes by Trey Ratcliff ( Stuck in Customs on flickr)

The Alien Garden I Found on that Away Mission by Trey Ratcliff ( Stuck in Customs on flickr)

Tokyo Road by Trey Ratcliff ( Stuck in Customs on flickr)

Here’s the code, notice that you need to get an API key from the flickr api site:

<?
/**
  * This piece of code is created by Bård Aase <elzapp@gmail.com>, 
  *  and is licensed under the GPL.
  *  Contact me if you need another license. 
  */
function sc_flickr($atts){
    /**
     * This function gathers information about a flickr image, and complies with the
     * shortcode API in Wordpress.
     * @param $atts array of options
     *   Options:    pid: picture id
     *               version: Small, Medium or Large (default:Medium)
     *               class: class to apped to the <div class="image "> so you can style it
     *               url: You can use an URL instead of a picture id, the pid will be 
     *                     extracted
     */
    $FLICKR_APIKEY="INSERT YOUR API KEY HERE"
    extract(shortcode_atts(array(
    "pid" => $atts[0],
    "version"=>"Medium", //setting default value
    "class"=>"centered", //setting default value
    "url"=>""
    ), $atts));
    if($url != ""){
       $e=explode("/",$url);
       $pid=$e[5];
    }
    /* broken into several lines to look better on the blog */
    $sizes_url="http://api.flickr.com/services/rest/?method=flickr.photos.getSizes";
    $sizes_xml=file_get_contents("{$sizes_url}&api_key={$FLICKR_APIKEY}&photo_id=$pid");
    $info_url="http://api.flickr.com/services/rest/?method=flickr.photos.getInfo";
    $info_xml=file_get_contents("{$info_url}&api_key={$FLICKR_APIKEY}&photo_id=$pid");
    $c=new SimpleXMLElement($info_xml);
    $sizes=new SimpleXMLElement($sizes_xml);
 
    $size=$version;
    $un=$c->xpath("//owner/@username");
    $rn=$c->xpath("//owner/@realname");
    $t=$c->xpath("//title");
    $u=$c->xpath("//url");
    $image=$sizes->xpath("//size[@label='$size']/@source");
    $width=$sizes->xpath("//size[@label='$size']/@width");
    $height=$sizes->xpath("//size[@label='$size']/@height");
    $fontsize="A";
    if($height[0] > 300 & $width[0] > 500){
        $fontsize="C";
    }elseif($height[0] > 200 & $width[0] > 300){
        $fontsize="B";
    }
    $r=<<<EOT
<div class="image imageclass{$fontsize} {$class}" style="width:{$width[0]}px">
<img src="{$image[0]}" width="{$width[0]}" height="{$height[0]}"><div class="caption">
<span class="title"><a href="{$u[0]}">{$t[0]}</a></span> 
    <span class="author">by {$rn[0]} ( 
    <a href="http://flickr.com/photos/{$un[0]}">{$un[0]}</a> on flickr)</span>
</div>
</div>
EOT;
    return $r;
}
?>

Put that in a file in your theme directory, and call the file “flickr.php”, then put the following into the functions.php-file in the same folder:

require_once("flickr.php");
add_shortcode('flickr','sc_flickr');

Note that this generates the HTML only, the visual style is added to your blog's css, here's the CSS I've used in this example:

 
.image { position:relative; }
.caption {
  position:absolute;
  bottom:0;
  left:0;
  right:0;
  padding:5px;
  background:transparent url(tr75b.png) repeat scroll left top;
  color:#FFF;
  font-family:"Trebuchet MS";
}
.image:hover .caption {
  background:#000;
}
.caption a {
  color:#CFC;
}
.image img {
  display:block;
}
.imageclassA { font-size:10px;}
.imageclassB { font-size:14px;}
.imageclassC { font-size:18px; }
.image.centered {
margin:auto;
}
.image.floatleft {
margin:5px;float:left;
}
.image.floatright {
margin:5px;float:right;
}

the image referenced in the stylesheet is available from here

Lets say you have two lists, one of them containing the values [1,2,3] and the other one containing the values [4,5,6], and you want to combine these two lists into one list, like this one: [1,2,3,4,5,6].

Let’s see how this can be done in different (high level) programming (or scripting or templating) languages.

Java

Java has multiple list implementations, lets start with the most basic one, the builtin array.

int[] A = {1,2,3};
int[] B = {4,5,6};
int[] C = new int[A.length + B.length];
int counter=0;
for(int e : A){
  C[counter]=e;
  counter++;
}
for(int e : B){
  C[counter]=e;
  counter++;
}

or with an ArrayList:

/*Dropped importing the correct classes, 
and constructing the ArrayLists A and B, since thats not the focus here...*/
List<Integer> C = new ArrayList();
C.addAll(A);
C.addAll(B);

PHP

So, how is the same done in PHP?

$A=array(1,2,3);
$B=array(4,5,6);
$C=array_merge($A,$B);

Python

And at last, how does the Pythonistas do this?

A=[1,2,3]
B=[4,5,6]
C=A + B

Note: the rubyists use the same approach.

As a programmer and linux geek I often need, or at least want to write about code, scripts and commands on my blog, but as many of you know, displaying raw code in HTML is setting yourself up for failure. The simplest part of the problem is the fact that line breaks and whitespace might be significant, this is simply solved by putting the code inside a <pre> block. The more tricky part is that code often is full of characters like &, > and <. Characters that have a specific meaning in HTML and can’t be echoed directly. & needs to be typed as an HTML entity (&amp;), just as > (&gt;) and < (&lt;). Talking about HTML and XML code on your blog can be a really tedious task (as the above).

I have usually solved this by writing the code in a file first, and then running the file through this sed command

cat file.html|sed -e 's/&/&amp;/' -e 's/</&lt;/' -e 's/>/&gt;/'

But as I work on a Macintosh these days, I’ve been using TextMate a bit.. Not as much as i use OpenKomodo though. TextMate has some nice blogging capabilities. You can use it to post directly onto your blog, and it has shortcuts for converting sections of your post into HTML entities. Just select the text, and hit ⌘+&, 1

Edit: it seems like there is some compatibility problems with WordPress 2.7 and TextMate. The post was published two hours off, and as a private post.

If you’re really trying, you will always be able to write completely unreadable code in any programming language. All the language designers attemts to force you to write readable code, and even making it so the easiest way ahead, even for quick hacks is to write it as readable and reusable code will fail if you’re really trying to write unreadable code.

This guy is obviously concerned about “job security”. (not the blog author, but the code author)

#!/usr/bin/env python
import os,sys
C=os.chdir
S=os.system
M=os.mkdir
J=os.path.join
A=os.path.abspath
D=os.path.dirname
E=os.path.exists
W=sys.stdout.write
V=sys.argv
X=sys.exit
ERR=lambda m:W(m+"\n")
PRNT=lambda m:W(m+"\n")
assert len(V)==2,"you must provide a sandbox name"
SB=V[1]
H=A(D(__file__))
SBD=J(D(H),SB)
C(SBD)
PST=J(SBD,'bin/paster')
VAR=J(SBD,'var')
ETC=J(SBD,'etc')
S("mkdir -p "+VAR)
PRNT("restarting "+SB)
CMD=";".join(['source %s'%J(SBD,'bin/activate'),PST+" serve --daemon --pid-file=%s/sandbox.pid --log-file=%s/sandbox.log %s/sandbox.ini start"%(VAR,VAR,ETC)])
PRNT(CMD)
S(CMD)
PRNT("All done!")
X(0)

Aaannd… Heres my understanding of the code:

#!/usr/bin/env python
import os,sys
 
#bail out if there's no sandbox specified
assert len(sys.argv)==2,"you must provide a sandbox name"
sandbox=sys.argv[1]
 
#get the dir of this script
home=os.path.abspath(os.path.dirname(__file__))
 
#sandboxdir is this dir/sandboxname
sandboxdir=os.path.join(os.path.dirname(home),sandox)
 
#go into the root of the sandbox
os.chdir(sandboxdir)
PST=os.path.join(sandboxdir,'bin','paster')
VAR=os.path.join(sandboxdir,'var')
ETC=os.path.join(sandboxdir,'etc')
 
#make var directory if it doesnt exists
if not os.path.exists(VAR):
    os.mkdir(VAR)
 
print "restarting "+sandbox
 
# Build the system commands required to restart the sandbox
cmds=[]
cmds.append('source %s' % os.path.join(sandboxdir,'bin','activate'))
cmds.append(PST+" serve --daemon --pid-file=%s/sandbox.pid"
            "--log-file=%s/sandbox.log %s/sandbox.ini start" % (VAR,VAR,ETC))
CMD=";".join(cmds)
PRNT(CMD)
 
#run the cmds
if os.system(CMD) == 0:
    print "All done!"
    sys.exit(0)
else:
    print "Ooops. failed."
    sys.exit(1)

Okay. It’s kind of stupid to put static typing up against Unit tests. Even so, lots of people does so a lot of times in the discussion between dynamic and static programming languages.

Static typing provides you with a security net when it comes to typos. Alhough, it doesn’t give you any security against logical errors.

Read the rest of this entry »

So, you’re going to parse a webpage, to extract some information. For instance if you want to get the tracking information for your last online order, and you want to display the tracking information changes using growl, dbus notifications or xosd.
You know regular expressions, so you go to the job with your long range missiles ready. But wait a minute, you’ll probably solve the problem but is regular expressions really the right tool?
The pro for regular expressions is that you can use the same tool you always use for parsing jobs, but then again you doesn’t learn anything new out of this. You might fortify your position as regex wizard even more, but how about something completely different?

Now. Most webpages is written in HTML, and some even in XHTML, for HTML documents languages like Python has a built-in parser, after the model of the SAX-parser. (It’s probably the other way around, the SAX parser is built on the base of the HTML parser…) Most programming languages has good support for XML, so for XHTML documents, you can use the HTML-parser, a SAX-parser or even the XML-DOM parsers.

The benefit of doing it this way is that your parser will probably be more robust to minor changes in the webpage. You don’t reinvent the wheel (The best way I’ve found to parse HTML documents using regular expressions is to make a specialized SAX-like parser anyway). Your code will probably be readable in a year, and others might even be able to understand your code. And finally, you learn something new, which might give you a fresh view on a lot of problems.

Now back to the original issue, to make a parser for the parcel tracking of your postal service. Here’s an example parsing the shipment tracking page of posten, the norwegian postal service.

Read the rest of this entry »

Currently I’m developing a webservice interface for queue handling in Asterisk towards another company, that is developing a CRM solution which will utilize my webservice for Computer Telephony Integration (CTI).

So, today the developer in the other company got such an understanding of the service that he started suggesting changes. A lot of changes in the DTD etc. These changes would imply some changes in the architecture downwards in the system. It is quite scary to start tearing things apart, when you already got something working. That is, if you don’t have a large amount of unittests.

Luckily I had good unittest coverage. So changing stuff was really just a matter of changing the tests, do fit the proposed changes. Test to see where I needed to change something. Do the changes and run the tests again.

It feels very good to have proper knowledge that everything actually works properly after a refactoring like this.

Unit testing is like love. It’ll keep you going when everything else fails. (citation)