JSONP. Compromised in 3…2…1…
To embed a vimeo video on some page, I had a look at their different methods for embedding and the easiest one seemed to be what is basically JSONP - a workaround for the usual restriction of disallowing AJAX over domain boundaries.
But did you know, that JSONP not only works around the subdomain restriction, it basically is one huge cross site scripting exploit and there's nothing you can do about it?
You might have heard this and you might have found articles like this one thinking that using libraries like that would make you save. But that's an incorrect assumption. The solution provided in the article has it backwards and only helps to protect the originating site against itself, but it does not help at all to protect the calling site from the remote site.
You see, the idea behind JSONP is that you source the remote script using <script src="http://remote-service.example.com/script.js"> and the remote script then (after being loaded into your page and thus being part of your page) is supposed to call some callback of the original site (from a browsers standpoint it is part the original site).
The problem is that you do not get control over the loading let alone content of that remote script. Because the cross-domain restrictions prevent you from making an AJAX request to a remote server, you are using the native HTML methods for cross domain requests (which should not have been allowed in the first place) and at that moment you relinquish all control over your site as that remotely loaded script runs in the context of your page, which is how you get around the cross domain restrictions - by loading the remote script into your page and executing it in the context of your page.
Because you never see that script until it is loaded, you cannot control what it can do.
Using JSONP is basically subjecting yourself to an XSS attack by giving the remote end complete control over your page.
And I'm not just talking about malicious remote sites... what if they themselves are vulnerable to some kind of attack? What if they were the target of a successful attack? You can't know and once you do know it's too late.
This is why I would recommend you never to rely on JSONP and find other solutions for remote scripting: Use a local proxy that does sanitization (i.e. strict JSON parsing which will save you), rely on cross-domain messaging that was added in later revisions of the upcoming HTML5 standard.
Sense of direction vs. field of view
Last saturday, I bought the Metroid Prime Triloogy for the Wii. I didn't yet have the Wii Metroid and it's impossible for me to use the GameCube to play the old games as the distance between my couch and the reciever is too large for the GameCube's wired joypads. It has been a long while since I last played any of the 3D Metroids, and seeing the box in a store made me want to play them again.
So all in all, this felt like a good deal to me: Getting the third Prime plus the possibility to easily play the older two for the same price that they once asked for the third one alone.
Now I'm in the middle of the first game and I made a really interesting observation: My usually very good sense of direction seems to require a minimum sized field of view to get going: While playing on the GameCube, I was constantly busy looking at the map and felt unable to recognize even the simplest landmarks.
I spent the game in a constant state of feeling lost, not knowing where to go and forgetting how to go back to places where I have seen then unreachable powerups.
Now it might just be that I remember the world from my first playthrough, but this time, playing feels completely differently to me: I constantly know where to go and where I am. Even with rooms that are very similar to each other, I constantly know where I am and how to get from point a to point b.
When I want to re-visit a place, I just go there. No looking at the map. No backtracking.
This is how I usually navigate the real world, so after so many years of feeling lost in 3D games, I'm finally able to find my way in them as well.
Of course I'm asking myself what has changed and in the end it's either the generally larger screen size of the wide-screen format of the Wii port or maybe the controls via the Wiimote that feel much more natural: The next step for me will be to try and find out which it is by connecting the Wii to a smaller (but still wide) screen.
But aside of all that, Metroid just got even better - not that I believed that to be possible.
Programming languages names
Today in the office, a discussion about the merits of Ruby compared to Python and the other way around (isn't it fun to have people around actually willing to discuss such issues?) lead into us making fun of different programming languages by interjecting some sore points about them into their names.
The Skype conversation went roughly as follows (I removed some stuff for brevity but all the language names are intact):
thepilif: ja-long variable names and no function pointers-va really sucks
thepilif: though there's always C(*^~**<<)++
thepilif: and then there's alyways Del-Access violation at address 02E41C10. Read of address 02E41C10-phi
thepilif: or P-false==true-HP
Coworker: ok so for the sake of it i should add py thon
thepilif: or java-everything is global-script
thepilif: too bad it doesn't work for C
thepilif: C-sigsegv
thepilif: they know why they just chose one letter
Coworker: exactly, k&r are smart
Coworker: has-how the fuck do i do a print-skell?
Coworker: pe/(^$^)/rl
thepilif: or pe-module? object? hash? what's the difference-rl
Coworker: so we could say pe/$^/rl
thepilif: and ru-lets rewrite our syntax on the fly-by
Coworker: l(i(s(p)))
thepilif: can't you wrap this into another pair of ()?
thepilif: (l(i(s(p))))))
Coworker: yes even better
thepilif: and add the syntax error
thepilif: one too many )
Coworker: it's impossible to match them just by looking
thepilif: totally impossible. yes
Coworker: the human brain is no fucking pushdown automata
Coworker: but maybe the lisp people are
Coworker: vb! vb needs one
thepilif: visual-on error resume next-basic
thepilif: and of course brain-<<<<<******<<<>>>>-fuck
thepilif: c-tries to be dynamic, but var just doesn't cut it-#
thepilif: c-not quite java nor c(++)?-#
thepilif: though the first one feels better
thepilif: oh.. and of course HT-unknown error-ML
thepilif: as a tribute to IE6
thepilif: and of course la-no bugs but still not usable-tex
thepilif: sorry, Knuth
thepilif: and send-$*$_**^$$$-mail
So the question is: Do you have anything to add? Do you feel that we were overly unfair?
Introducing sacy, the Smarty Asset Compiler
We all know how beneficial to the performance of a web application it can be to serve assets like CSS files and JavaScript files in larger chunks as opposed to smaller ones.
The main reason behind this is the latency incurring from requesting a resource from the server plus the additional bandwidth of the request metadata which can grow quite large when you take cookies into account.
But knowing this, we also want to keep files separate during development to help us with the debugging and development process. We also want the deployment to not increase too much in difficulty, so we naturally dislike solutions that require additional scripts to run at deployment time.
And we certainly don't want to mess with the client-side caching that HTTP provides.
And maybe we're using Smarty and PHP.
So this is where sacy, the Smarty Asset Compiler plugin comes in.
The only thing (besides a one-time configuration of the plugin) you have to do during development is to wrap all your <link>-Tags with {asset_compile}....{/asset_compile} and the plugin will do everything else for you, where everything includes:
- automatic detection of actually linked files
- automatic detection of changed files
- automatic minimizing of linked files
- compilation of all linked files into one big file
- linking that big file for your clients to consume. Because the file is still served by your webserver, there's no need for complicated handling of client-side caching methods (ETag, If-Modified-Since and friends): Your webserver does all that for you.
- Because the cached file gets a new URL every time any of the corresponding source files change, you can be sure that requesting clients will retrieve the correct, up-to-date version of your assets.
- sacy handles concurrency, without even blocking while one process is writing the compiled file (and of course without corrputing said file).
sacy is released under the MIT license and ready to be used (though it currently only handles CSS files and ignores the media-attribute - stuff I'm going to change over the next few days).
Interested? Visit the project's page on GitHub or even better, fork it and help improving it!
Twisted Tornado
Lately, the net is all busy talking about the new web server released by FriendFeed last week and how their server basically does the same thing as the Twisted framework that was around so much longer. One blog entry ends with
Why Facebook/Friendfeed decided to create a new web server is completely beyond us.
Well. Let me add my two cents. Not from a Python perspective (I'm quite the Python newbie, only having completed one bigger project so far), but from a software development perspective. I feel qualified to add the cents because I've been there and done that.
When you start any project, you will be on the lookout for a framework or solution to base your work on. Often times, you already have some kind of idea of how you want to proceed and what the different requirements of your solution will be.
Of course, you'll be comparing existing requirements against the solutions around, but chances are that none of the existing solutions will match your requirements exactly, so you will be faced with changing them to match.
This involves not only the changes themselves but also other considerations:
- is it even possible to change an existing solution to match your needs?
- if the existing solution is an open source project, is there a chance of your changes being accepted upstream (this is not a given, by the way).
- if not, are you willing to back- and forward-port your changes as new upstream versions get released? Or are you willing to stick with the version for eternity, manually back-porting security-issues?
and most importantly
- what takes more time: Writing a tailor-made solution from scratch or learning how the most-matching solutions ticks to make it do what you want?
There is a very strong perception around, that too many features mean bloat and that a simpler solution always trumps the complex one.
Have a look at articles like «Clojure 1, PHP 0» which compares a home-grown, tailor-made solution in one language to a complete framework in another and it seems to favor the tailor-made solution because it was more performant and felt much easier to maintain.
The truth is, you can't have it both ways:
Either you are willing to live with «bloat» and customize an existing solution, adding some features and not using others, or you are unwilling to accept any bloat and you will do a tailor-made solution that may be lacking in features, may reimplement other features of existing solutions, but will contain exactly the features you want. Thus it will not be «bloated».
FriendFeed decided to go the tailor-made route but instead of many other projects each day who go the tailor made route (take Django's reimplementations of many existing Python technologies like templating and ORM as another example) and keep using that internally, they actually went public.
Not with the intention to bad-mouth Twisted (though it kinda sounded that way due to bad choice of words), but with the intention of telling us: «Hey - here's the tailor-made implementation which we used to solve our problem - maybe it is or parts of it are useful to you, so go ahead and have a look».
Instead of complaining that reimplementation and a bit of NIH was going on, the community could embrace the offering and try to pick the interesting parts they see fitting for their implementation(s).
This kind of reinventing the wheel is a standard process that is going on all the time, both in the Free Software world as in the commercial software world. There's no reason to be concerned or alarmed. Instead we should be thankful for the groups that actually manage to put their code out for us to see - in so many cases, we never get a chance to see it and thus lose a chance at making our solutions better.
Snow Leopard and PHP
Earlier versions of Mac OS X always had pretty outdated versions of PHP in their default installation, so what you usually did was to go to entropy.ch and fetch the packages provided there.
Now, after updating to Snow Leopard you'll notice that the entropy configuration has been removed and once you add it back in, you'll see Apache segfaulting and some missing symbol errors.
Entropy has not updated the packages to snow leopard yet, so you could have a look at PHP that came with stock snow leopard: This time it's even bleeding edge: Snow Leopard comes with PHP 5.3.0.
Unfortunately though, some vital extensions are missing, most notably for me, the PostgeSQL extension.
This time around though, Snow Leopard comes with a functioning PHP development toolset, so there's nothing stopping you to build it yourself, so here's how to get the official PostgreSQL extension working on Snow Leopard's stock php:
- Make sure that you have installed the current Xcode Tools. You'll need a working compiler for this.
- Make sure that you have installed PostgreSQL and know where it is on your machine. In my case, I've used the One-click installer from EnterpriseDB (which persisted the update to 10.6).
- Now that Snow Leopard uses a full 64bit userspace, we'll have to make sure that the PostgreSQL client library is available as a 64 bit binary - or even better, as an universal binary.Unfortunately, that's not the case with the one-click installer, so we'll have to fix that first:
- Download the sources of the PostgreSQL version you have installed from postgresql.org
- Open a terminal and use the following commands:
% tar xjf postgresql-[version].tar.bz2 % cd postgresql-[version] % CFLAGS="-arch i386 -arch x86_64" ./configure --prefix=/usr/local/mypostgres % make
make will fail sooner or later because you the postgres build scripts can't handle building an universal binary server, but the compile will progress enough for us to now build libpq. Let's do this:
% make -C src/interfaces % sudo make -C src/interfaces install % make -C src/include % sudo make -C src/include install % make -C src/bin % sudo make -C src/bin install
- Download the php 5.3.0 source code from their website. I used the bzipped version.
- Open your Terminal and cd to the location of the download. Then use the following commands:
% tar -xjf php-5.3.0.tar.bz2 % cd php-5.3.0/ext/pgsql % phpize % ./configure --with-pgsql=/usr/local/mypostgres % make -j8 # in case of one of these nice 8 core macs :p % sudo make install % cd /etc % cp php.ini-default php.ini
- Now edit your new php.ini and add the line
extension=pgsql.so
And that's it. Restart Apache (using apachectl or the System Preferences) and you'll have PostgreSQL support.
All in all this is a tedious process and it's the price us early adopters have to pay constantly.
If you want an honest recommendation on how to run PHP with PostgreSQL support on Snow Leopard, I'd say: Don't. Wait for the various 3rd party packages to get updated.
OpenStreetMap
The last episode of FLOSS Weekly consisted of an interview with Steve Coast from OpenStreetMap. I knew about the project, but I was of the impression that it was in its infancy both content-wise and from a technical perspective.
During the interview I learned that it's surprisingly complete (unless, of course, you need a map of Canada it seems) and highly advanced from a technical point of view.
But what's really interesting is the fact how terribly easy it is to contribute. For smaller edits, you just click the edit-Link and use the Flash editor to paint a road or give it a name. If you need or want to do more, then there's a really easy to use Java based editor:
First you drag a rectangle onto a pre-rendered version of the map which will cause the server to send you the vector information consisting of that part and then you can edit whatever you want.
If you have them, you can import traces of a GPS logger to help you add roads and paths and when you are finished, you press a button and the changes get uploaded and will be visible to the public a few minutes later (though one modification I made took about an hour to arrive on the web).
When the same nodes where updated in the meantime, a really nice conflict resolution assistant will help you to resolve the conflicts.
For me personally, this has the potential to become my new after-work time sink as it combines quite many passions of mine:
- The GPS tracking, importing and painting of maps is pure technology fun.
- Actually being outside to generate the traces is healthy and also a lot of fun
- Maps also are a passion of mine. I love to look at maps and I love to compare them to my mental image of the places they are showing.
And besides all that, Open Street Map is complete enough to be of real use. For biking or hiking it even trumps Google Maps by much.
Still, at least near where I live, there are many small issues that can easily be fixed.
As the different editors are really easy to use, fixing these issues is a lot of fun and I'm totally seeing myself cleaning out all small mistakes I come across or even adding stuff that's missing. After all, this also provides me with a very good reason to visit the places where I grew up to complete some parts.
The whole concept behind being able to update a map by just a couple of mouse clicks is very compelling too as it finally gives us the potential to have really accurate maps in a very timely fashion. For example: Last October, one of the roads near my house closed and just recently the tracks of the Forchbahn were moved a bit.
Just today I added these changes to OpenStreetMap and now OSM is the only publically available map that correctly shows the traffic situation. And all that with 15 minutes of easy but interesting work.
For those interested, my Open Street Map user profile is, of course, pilif.
SMS is dead
BeejiveIM is the first multiprotocol IM application for the iPhone that supports the new background notification features of firmware 3.0. Yesterday I went ahead and bought that application, curious to see how well it would work.
And just now my phone vibrated and on the display, there was an IM message a coworker sent me via Google Talk. The user experience was exactly the same as it would have been with an SMS - well - nearly the same - the phone made a different sound.
So the dream I had many moons ago (6 years - boy - how time flies) has finally come true, with one difference: Whereas back then the MB cost CHF 7, now it's practically free, considering that I'm unable to actually use up my traffic quota and even then, it's only CHF 0.10 now.
So let's keep that in mind and also consider that SMS pricing hasn't changed in the last six years.
So while IM was 52 times cheaper than SMS back then, now the price advantage ranges from somewhere between 3500 times cheaper and infinity times cheaper.
SMS pricing needs to be looked at. This just cannot be.
Of all the hardware that can break…
... it has to be the one that's most difficult to replace.
Today, my Gefen HDMI over Cat5 adapter died. Well. It didn't die completely, it just lost its ability to produce a stable image. What is transmitted is very intermittent and in the few seconds the image is available, it's heavily distorted.
Also, it's not the obvious issue (faulty cabling) as the problems did not go away after using two very short (1m) cat 5 cables to test.
Now this is really bad for a variety of reasons:
- Only just last Saturday I bought Star Ocean and Tales of Vesperia for my 360, giving me a total play time of 1.5 hours so far.
- Yesterday I noticed that Worms: Armageddon was released for Xbox arcade and I have already invited Ebi after the huge success that was our earlier Worms evening on the 360.
- My setup is totally dependent on the two extenders as I am covering more than 20 meters of distance between receiver and projector. No extender, no Xbox, no Wii, no projector.
- Last time I waited around six weeks for the extender to arrive
Of all the hardware I'm having at home, the HDMI extender is the worst to break. Not only is it very hard to replace (see above), it's so deeply integrated into my home cinema setup that just debugging what was going on took a ladder, a screwdriver, a hex-wrench and unwinding an ungodly heap of cables.
All of that in an apartment whose temperature is currently at 30°C (86 °F) and with a hell of a headache.
I'd take anything else going down. Anything but that Gefen extender. My XBox? Sure. Shion? It'd suck, but sure if it has to be, go ahead. My reciever? That would hurt as it was very expensive, but at least it's easily replaced.
Why did it have to be that Gefen extender? Why??