Thursday, July 15, 2010

GNUmed web interface - state of affairs

Back then there was the idea of a web interface for GNUmed. Back then there
was no clear idea of what that meant. I whipped together a few tutorial bits
of cherrpy and connected it to gmPG. This enabled us to log into the GNUmed
database and ran a few queries against the database to show there actually was
a connection. Little did I know that this was a beginning but nowhere near any
of what I wanted.

I wanted:
1.) a web client that could run concurrently to the wxpython client
2.) a web client that ran on top of gmPG2 instead of some framework's ORM
3.) role based security instead of database based security - no credentials in
a database table, period
4.) a web client that could fetch data from a backend
5.) not a web client that was no more then a number of server-side assembled
html pages.
6.) A nice looking web client without having to learn Javascript
7.) maybe more but I cannot remember

I checked loads of frameworks. But there was always a catch. The frameworks
wanted to do the work for me and they had a certain idea how to do it. Their
ideas did not neccessarly correlate with my ideas.

I ran my ideas by a couple of framework developers and Javascript toolkit
developers but none seems overly interested apart from their framework would
do what I wanted. Problems was I did not have enough skills and knowledge so
it was more a matter of trust then a fact based decision.

I always came across pyjamas. It seemed to partly fit the bill as it would let
em write python instead of Javascript. I ran my ideas by Luke Leighton after
seeing a message on python-announce mailing list.

That changes quite a few things. First he told me what I did not know about
the web. Most of the stuff was obvious for a web developer. I learned that the
web is stateless and unlike in a desktop application it is quite a bit of work
to make a client and a server process holding a unique persistent database
connection. Luke published quite a bit of code in easy to digest chunks so I
was able to follow along.

In the end here is what he solved for GNUmed.

The code:
1.) allows a browser (web client) to hold a persistent database connection to
the GNUmed backend even if the user closes the browser. The client will
reconnect to its unique connection when it is restarted.

2.) talks to the GNUmed database through the GNUmed middleware. No ORM or
stuff needed.

3.) demonstrated how two simultaneous connections (clients) talk to the
database without getting in each other's way or stealing each other's data
(which is common if you don't take care of it)

4.) provides a pyjamas application which will let a user log in and run
database queries.

5.) runs without interfering with other wxpython clients

6.) shows nicely how to use JSON-RPC to talk to client side frameworks. This
even lets Javascript framework warriors write their own client in Javascript
without having to do any database stuff.

The problems Luke solved were to do with turning HTTP connections,
which are ordinarily stateless, into permanent stateful connections.
He achieved this by:

* creating a special non-blocking HTTP proxy that does HTTP 1.1 with
"keep-alives", even if the browser does not.

* creating a modified version of python's SimpleHTTPServer that always
responds with "Connection: keep-alive", for GET and POST queries, even
on HTTP errors.

* inserting a session cookie into the HTTP stream, which the
non-blocking HTTP proxy looks out for, and uses to match the incoming
HTTP request with the corresponding outgoing connection.

in this way, browsers appear to be connected to the same persistent
back-end process, which is kept open and alive on behalf of the
browser, by the special proxy. as long as the back-end process
doesn't crash (or deliberately exit after a timeout) and as long as
the user does not clear the browser's cache, the user will always be
connected to the exact same back-end web server process.
What we have now is an excellent starting point to develop a web client for
GNUmed. Users have the option to run the wxpyton client or web client or both
at the same time. The big benefit of all client talking to the backend through
a common server side middleware is that in case of backend changes or
middleware changes all client continue to work without having to port all
changes to all clients.

Buyer beware: This is proof of concept code. It shows the concept works. Not
more. No less.

Rock on.


1 comment:

Greg said...

Awesome. Thanks for posting on the pyjamas mailing list. I hope you get this working because it would help out both projects: gnumed and pyjamas!