Compiling WebKit on Windows

1. If you're doing any open source programming, you'll need to constrain yourself to the Cairo-based WebKit, as Apple's CoreGraphics-based rendering is non-redistributable.
2. You'll need VS 2005, as anything newer isn't supported. If you're using Express edition, make sure you have the Windows Platform SDK.
   
3. You'll need cygwin to run the build scripts. Make sure you install perl (in cygwin) -- but not GCC, as this may interfere with the scripts finding Visual Studio's compiler.
4. If cygwin has any problems with "/r" character in text files, run the command "dos2unix $(find . -name '*.sh' -o -name '*.pm')" in "C:\cygwin\home\$username", to convert bash and perl files to UNIX new-lines.
   
5. Run "C:\Program Files\Microsoft Visual Studio 8\VC\bin\vcvars32.bat" to enable the path's needed to run the compiler from command line.
   
6. Check out your source from SVN. Nightly tarballs have some sort processing step specific to the platform that may remove things you might want.
7. Check out the SVN into "C:\cygwin\home\$username", as the build scripts assume this.
8. Set the environment variables "WEBKITLIBRARIESDIR=C:\cygwin\home\$username\WebKitSupportLibrary\win" and "WEBKITOUTPUTDIR=C:\cygwin\home\$username\webkit\WebKitBuild".
9. Configure WebCore/config.h to build with Cairo instead of "Safari", as according to this.
10. Run WebKit/WebKitTools/Scripts/update-webkit
11. Run WebKit/WebKitTools/Scripts/update-webkit-support-libs
12. Run WebKit/WebKitTools/Scripts/update-webkit-auxiliary-libs
13. Run WebKit/WebKitTools/Scripts/build-webkit --cairo-win32 --debug
14. To be continued!
    

Mini book reviews

I buy and read a lot of computer books (for various definitions of "read" -- references often aren't page turners).

I always have my judgement about the book from my point of view, but I hesitate to share it with others, as I think there is no better opinion than the one a reader has after he's read the whole book on his own.

That said, when I buy books, I want to read what others thought of it. So to provide a service to my faithful readers, I am kicking off some mini-reviews of book's I've read, mostly in relation to my work.

About Face 3: Essentials of Interaction Design

For anyone like me, who hasn't formally studied Human-Computer Interaction in school but now finds themselves working on anything with a non-trivial user interface, this is the first book I recommend you read.

It's extremely easy to read, and spends a respectable part of the beginning of the book explaining why UIs need to be designed, before moving on to the how of designing one. If you're buying an interaction design book it's because you realize the need for UI design yourself, but may be first required to convince management that the interaction design process needs to be taken seriously; the beginning section will give you that ammunition.

However those who are already professional designers will likely find it fairly a waste of time, and spend most of the book skimming over repetitions of things they already know.

Those like myself, who feel they have a bit of intuition for user interaction design, will find a fair amount to skim over; however without any previous training HCI, it is reassuring to have your intuitions confirmed by the experts, and so even the skimming will be rewarding.

The reason why I chose this book over others is that it's a tutorial, and it proposes a simple design method that concentrates on the heart of designing any tool: people (personas) and what they want to do (goals). It's simple, direct, and resonates very well with my design sensibilities.

The middle part of the book introduces the core methodology (Goal-directed Design), and the latter part of the book moves on to placing that theory within the context of contemporary GUI design. An excellent combinational of motivation, leading into theory, leading into method, leading into example, in a comprehensive, intuition building manner.

Any software developer without existing library on user interaction design needs this book in their core library.

Content Networking: Architecture, Protocols, and Practise

I am not sure if the book defines "Content Networking" very well, but basically it amounts to "how do you get lots of cool content to lots of people on an Internet Scale?" This is very relevant question for our virtual world design efforts.

Printed 2005, this book is somewhat dated given that I am basically only looking out for state-of-the-art texts (or theory, which is undying).

It has chapters on web content, streaming media, IM, and P2P, but those chapters, for me, feel like very cursory survey chapters that don't tell me very much. I enjoyed reading them and found them useful, but I'd probably rather buy dedicated books on each of those topics separately.

What I found very interesting was the treatment of web content, and the secret weapon of the web that makes it so insanely scalable: load balancing and caching. Both of which are relatively simple techniques, but the book goes into specific methods in detail and really opened my eyes to what goes on that I merely took for granted. HTTP gets static for having higher overhead than UDP, but this book makes it pretty clear the stateless nature of HTTP means you can do amazing things with caching and load balancing that make the web scale to global levels.

If you're planning a large scale content deployment system I recommend taking a browse through this survey book, but otherwise it's not overly impressive. Borrow a copy.

Parallel and Distributed Simulation Systems

From the title it's appropriateness to virtual worlds is obvious. However its printing date of 2000 belies a different focus. The book assumes an audience of mostly military simulation designers, and makes direct reference to various standards published by the US military for use in war simulations. The book makes reference to "casual" uses outside tanks and jets, and may even use the phrase "virtual world", but it will take some imagination to remove the discussion from it's military context and place it in a VW one.

The book has a distinct academic feel, visible in its eagerness to categorize all different kinds of methods or system types and give them "helpful" acronyms. The reading is a bit dry, but not overly technical.

If you are looking for a systematic approach to simulations, and want to know what the baseline standards are, having been established in 2000 or earlier, it's worth spending some time to look through. It is probably reassuring to know some terminology within the book, and that you can't go wrong doing something the military has done previously. However there are no new or interesting ideas here at all. At best a reference for a fall-back implementation.

Level of Detail for 3D Graphics

Level of Detailing is quite possibly one of the single most critical linchpins for scalability in a virtual world. You can have a large number of objects sitting in a database, but at the end of the day, a camera sees everything up until the horizon, in all directions, and in some manner everything visible has to be brought into memory and drawn. Clearly the less work "drawing" all those things are, the higher your simulation can scale.

Sometimes reading a book is less about what it tells you as what it doesn't tell you, or rather what it tells you "don't worry, you're not missing anything":

Although the book is dated, the book told me the state of the art LoDing in 2003 boils down to the following decisions: when to LoD, what to LoD, and how far to LoD. LoDing itself is taken for granted as only mesh simplification, which was a bit of a disappointment in that I was hoping there were more magical techniques available, especially for textures which are a substantial burden in VWs. Perhaps there is more to LoDing, just all of it invented post 2003. I was also hoping to find some techniques that minimize loading of meshes, but perhaps the authors failed to consider anyone would be so reckless as to download meshes over the internet. :)

Disappointments aside, mesh simplification is extremely important for future reX, but I suspect that loading those meshes from over the network is by far the greater bottleneck, hitting long before the graphics card breaks a sweat from excessive vertices, so for us mesh simplification would have to happen before the client viewer downloads it.

Recommended for anyone who is implemented a LoD system where huge meshes are bottle necking the G/CPU.

Treating RSI

One of the most critical health issues around programming is RSI. If not properly addressed, it can end your career. For a while I was quite concerned that it would end mine.

The problem with RSI is it seems the causes are each subtly different, and so it responds differently to various treatments.

Here then is my personal set of treatments. If you're feeling any discomfort at all, don't wait, change your set-up immediately.

• See a doctor immediately. I eventually got a doctor to drain the cyst in my wrist, drastically reducing the discomfort.
  
• Take breaks of all kinds. Hourly breaks for 5 minutes. Lunch breaks. Weekends off. Long computerless vacations.
• Sleep. Highly underrated treatment. Directly proportional to comfort.
• Wear long-sleeve shirts and even fingerless gloves in order to keep the hands warm while working. Also very underrated.
• Ice inflamed areas after work.
• Kinesis Advantage keyboard. Worth more to me than the entire computer itself.
• Ergonomic mouse. Try to never use it. Especially the wheel.
• Turn up the thermostat in your computer room. Never program in the cold.
• Sleep. Some more.
  

Asterisk Realtime Architecture

Like any machine tinkered with heavily over time, Asterisk has a lot of exposed configuration points in a lot of places, and it can be hard to know how or why what you want to do isn't working because you neglected to set some variable that became necessary since the last time the module was documented.

I've fallen into this situation for the past couple days, and having finally found the right thing to tweak, let me write down how to use ODBC to access Asterisk's configuration files, including the dial plan, from a real-time database.

First thing: go read the official book by O'Reilly, and read the chapter on ARA.

Once you have downloaded Asterisk, as we did here, run ./configure, then make menuselect and ensure res_odbc, res_config_odbc, and all *_realtime modules are set
to be built. If they have a 'XXX' beside their name, it means that you need to install some dependencies. Search your package installer for "ODBC" and a database that supplies an ODBC driver, such as PostgreSQL or MySQL. You may also need the "-devel" version of those packages.

To make the tutorial complete, we'll pick a DB for the examples, so we choose PostgreSQL.

1. Setting Up the Database

As the DB's (not the system's) root user create a new role and database for Asterisk to use:
createuser -SRdP asterisk-role
createdb --owner=asterisk-role asterisk-db

Then restart the database.

2. Setting Up ODBC

Ensure that /etc/odbcinst.ini exists and contains something like the following:
[PostgreSQL]
Description = ODBC for PostgreSQL
Driver = /usr/lib/libodbcpsql.so
Setup = /usr/lib/libodbcpsqlS.so
FileUsage = 1

Ensure that /etc/odbcinst.ini exists and contains something like the following:
[asterisk-postgres-connector]
Description = PostgreSQL connection to 'asterisk' database
Driver = PostgreSQL
Database = asterisk-db
Servername = localhost
UserName = asterisk-role
Password = 123456

Then verify that the connection works using the following command:
echo "select 1" | isql -v asterisk-postgres-connector

3. Setting Up a Basic Asterisk Configuration

extconfig.conf
[settings]
sipusers => odbc,asterisk-postgres-connector,sip_table
sippeers => odbc,asterisk-postgres-connector,sip_table
extensions => odbc,asterisk-postgres-connector,extensions_table

extensions.conf
[general]
[default-sip]
switch => Realtime/@

modules.conf
[modules]
autoload=yes
preload => res_odbc.so
preload => res_config_odbc.so

res_odbc.conf
[ENV]

[asterisk-postgres-connector]
enabled => yes
dsn => asterisk-postgres-connector
username => asterisk-role
password => 123456
pre-connect => yes

sip.conf
[general]

4. Setting up Dialplan and Sip User/Peer Tables

This is perhaps the most mysterious point. The ARA requires a number of columns to be set within the table, and if they aren't it will silently fail. The SIP table in particular. Nearest I can tell, the only source of documentation is here.

Run the following SQL code as the asterisk-role:

CREATE TABLE extensions_table
(
"id" serial,
"context" varchar(20) NOT NULL default '',
"exten" varchar(36) NOT NULL default '',
"priority" smallint NOT NULL default '0',
"app" varchar(20) NOT NULL default '',
"appdata" varchar(128) NOT NULL default '',

PRIMARY KEY (id)
) WITHOUT OIDS;

CREATE TABLE sip_table
(
"id" serial,
"name" varchar(80) NOT NULL default '',
"host" varchar(31) NOT NULL default 'dynamic',
"nat" varchar(5) NOT NULL default 'route',
"type" varchar(6) check (type in ('user','peer','friend')) NOT NULL default 'friend',
"accountcode" varchar(20) default NULL,
"amaflags" varchar(13) default NULL,
"callgroup" varchar(10) default NULL,
"callerid" varchar(80) default NULL,
"call-limit" int NOT NULL default '0',
"cancallforward" char(3) default 'no',
"canreinvite" char(3) default 'no',
"context" varchar(80) default 'default-sip',
"defaultip" varchar(15) default NULL,
"dtmfmode" varchar(7) default NULL,
"fromuser" varchar(80) default NULL,
"fromdomain" varchar(80) default NULL,
"insecure" varchar(4) default NULL,
"language" char(2) default NULL,
"mailbox" varchar(50) default NULL,
"md5secret" varchar(80) default NULL,
"deny" varchar(95) default NULL,
"permit" varchar(95) default NULL,
"mask" varchar(95) default NULL,
"musiconhold" varchar(100) default NULL,
"pickupgroup" varchar(10) default NULL,
"qualify" char(3) default NULL,
"regexten" varchar(80) default NULL,
"restrictcid" char(3) default NULL,
"rtptimeout" char(3) default NULL,
"rtpholdtimeout" char(3) default NULL,
"secret" varchar(80) default NULL,
"setvar" varchar(100) default NULL,
"disallow" varchar(100) default '',
"allow" varchar(100) default '',
"fullcontact" varchar(80) NOT NULL default '',
"ipaddr" varchar(15) NOT NULL default '',
"port" smallint NOT NULL default '0',
"regserver" varchar(100) default NULL,
"regseconds" int NOT NULL default '0',
"username" varchar(80) NOT NULL default '',
"delay" int NOT NULL default '0',
"sortorder" int NOT NULL default '1',
PRIMARY KEY (id),
UNIQUE (name)
) WITHOUT OIDS;

GRANT ALL ON sip_table TO asterisk;
GRANT ALL ON extensions_table TO asterisk;

INSERT INTO extensions_table (context, exten, priority, app, appdata) VALUES ('default-sip', 'sip-extn', 1, 'Answer', '');
INSERT INTO extensions_table (context, exten, priority, app, appdata) VALUES ('default-sip', 'sip-extn', 2, 'Wait', '2');
INSERT INTO extensions_table (context, exten, priority, app, appdata) VALUES ('default-sip', 'sip-extn', 3, 'Playback', 'hello-world');
INSERT INTO extensions_table (context, exten, priority, app, appdata) VALUES ('default-sip', 'sip-extn', 4, 'Wait', '2');
INSERT INTO extensions_table (context, exten, priority, app, appdata) VALUES ('default-sip', 'sip-extn', 5, 'Hangup', '');

INSERT INTO sip_table (name, context, host, type) VALUES ('sip-user', 'default-sip', 'dynamic', 'friend');

5. Test the setup

Run Asterisk on the console, and verify the ODBC connection is working by issuing the command odbc show.

Don't bother trying to run dialplan show or similar, as this will not display what we inserted in the previous step. Instead we need to make a call to test: in your soft-phone, dial sip:sip-extn@asterisk.example.com, and you should hear a voice say "hello world".

Working with Asterisk

Recently I have been working on some VoIP based solutions using Asterisk.

I've found it a bit hard to work on, partly because Asterisk appears to suffer from some ad-hoc growth over time, and not only is the design somewhat dated, but more importantly the documentation left around the web appears to be tied to past versions such that working with the newest versions can leave you scratching you head why what one guy did, and what you're doing don't seem to match up. Even when you manage to get the example working you're never quite sure which flag you twiddled happened to do the trick.

(Aside: there seems to be a lot of consulting business around Asterisk. Partly this is due to the inherently Enterprise-related nature of a PBX. However I have to wonder if these consultants don't have a vested interest in keeping Asterisk opaque. Certainly they'd have more clients than if the community spent time making it easier to use.)

First thing one needs to do when learning a system is to make some small examples. The easiest way to test your examples, lacking the sort of hardware one might have if they owned a real Enterprise phone system, is to use software phones. The most common kind of software phone is one that uses SIP to set up an audio stream (over RTP) for voice communication.

To obtain a working Asterisk server, download either the source, or a pre-compiled binary from your favorite location. If you're working a lot with Asterisk it makes sense to build the source yourself, since it's rather easy to build.

After you have completed the install, your /etc/asterisk/ directory should be full of a wild assortment of configuration files, each over-teeming with tweak-worth options. However just to set up a simple call, we don't need such complexity. Copy those files to a safe place, and consider the minimal files below:

modules.conf
As far as I can tell, this is all you'll ever need from modules.conf, under any situation.

[modules]
autoload=yes

This means that as Asterisk looks for .conf files (which we have mostly removed), and processes its internal and external dependencies, it will automatically load the binary files in /usr/lib/asterisk/modules.

sip.conf
This file controls the SIP input "channel", from which any session created from sip:a-sip-user@example.com to sip:somewhere@asterisk.example.com, must pass first on its way through an Asterisk system. The call from a-sip-user is assigned a "context" which is where in the "dialplan" the call should be routed. ("user" means that the call is in-bound only, and "dynamic" means the server should look up the user's IP address dynamically)

[a-sip-user]
context=my-sip-context
type=user
host=dynamic

This means that anyone who call into Asterisk, and declares themselves to be "a-sip-user" via their SIP URI (such as sip:a-sip-user@somewhere.example.com), is given a dial plan context of "my-sip-context".

extensions.conf
This is the "dial plan", the place where the routing logic of a PBX is stored. It consists of a number of named contexts, under which a series of sequential functions are called. It will appear odd to a modern programmer, but if one considers the evolution of old-fashioned hardware based PBXs, it makes more sense.

Basically there is first an "extension", which is related to the old extension numbers you would have to dial over an old-fashioned system. However with software based PBXs, those extensions can now easily be human-readable strings.

A call is brought in to the dial plan from a channel. The only channel we are considering now is a SIP-based VoIP call. That channel attached a context to the call, and starting from the named context, the dial plan attempts to match an extension pattern to the one specified in a call.

In the case of a SIP call, where the URI is sip:my-destination@asterisk.example.com, the extension is considered to be "my-destination".

[my-sip-context]
exten => sip-extn,1,Answer()
exten => sip-extn,n,Wait(2)
exten => sip-extn,n,Playback(hello-world)
exten => sip-extn,n,Wait(2)
exten => sip-extn,n,Hangup()

This means that when dropped in the "my-sip-context" context, Asterisk will attempt to match the extension against "sip-extn". The first action to be taken is to answer the call. The steps with priority marked "n" proceed sequentially from each other.

In the example above, the soft-phone must dial the following URI: sip:sip-extn@asterisk.example.com as the user designated sip:a-sip-user@somewhere.example.com. Upon doing so, they should hear a female voice saying "hello world!"

If you are using ekiga as your soft-phone, go to Edit->Accounts, and then add a new one. The name can be any thing you like, however the registrar must be asterisk.example.com (that is the name or IP address of the machine running Asterisk), and the user must be a-sip-user. The password is empty because we have not required password protection in sip.conf.

Click OK and confirm that the Status indicates that it is registered with the server. If registration fails, check that the server's firewall is not blocking port 5060, and that you are not using the soft-phone from the same machine that Asterisk is running on since port numbers will conflict.

In the top-most address bar, enter the URI sip:sip-extn@asterisk.example.com and click enter. You should now hear the words "hello world!"

The version of Asterisk I used was 1.6.