Sunday, June 7, 2009

On Platforms and the Convergence of 2D and 3D Internet

Working in Virtual Worlds, we are often given to wonderful discussions about what the future will hold for the Internet, and when and how various technologies will converge.

One popular school of thought is that virtual worlds should be embedded in existing web browsers as "light" plug-ins -- in order to make installation easy, and leverage history's most popular connected application platform.

The other is that there should be a "heavy" viewer designed completely around a unique virtual world experience; which happens to contain a superset of features, including web-browsing.

I belong to the second school; and in fact that opinion had lead to conflicts within my previous company. Thankfully, my current employer very much shares my vision.

However there are many in the first camp, and I am sometimes asked why I will not join them, especially in light of recent developments specifically aimed at bringing 3D to web browsers. Let me try to explain why despite all the momentum for adding 3D to web browsers:

I believe placing a 3D virtual world viewer within a 2D web viewer is somewhat comparable to embedding a gopher viewer, or ftp viewer, or email viewer within a web browser -- it's of course been done, but it's backwards, leads to awkward results, and has never proven to be very popular.

In the long term, computer code is plastic. If the world decided to morph a web browser into a 3D application platform overnight, there is no technological reason why it couldn't happen. So the question of where to put certain features is more ideological, metaphorical, or about user interaction concerns

What is a web browser?

Is it the ultimate convergence point for anything internet-related? Is it a new OS that should have it's own version of OpenGL? Is it just a safer sandbox for internet applications that the OS itself?

For me, issues of security or specific features are not interesting. Clearly any application platform needs to constrain the ways in which hosted applications can access raw resources. And clearly the needs of the platform evolve over time, and that requires adding new APIs.

For me, the question of what makes a web browser is a question of design, and mental modes; and how the totality of those decisions affect the expressiveness of applications written for the platform.

A web browser, for me, is a transaction-oriented 2D document-centred internet application platform. A virtual world viewer is a stream-oriented real-time interactive 3D scene-centred internet application platform. I am not sure there exists any easy abstraction that allows one to fluidly translate concepts between categories. The commonality that they host applications that run on the internet is not enough to unite disparate modes, and inform the design of a coherent platform.

Web browsers will indeed need 3D features. But they way in which 3D will be used on the web will be document-oriented, and happen in transactions (not real time). Things like e-commerce catalogues can be made available in full 3D. Any number of static scenes relating to the host web document will add a lot of value -- the way embedded streaming video adds value to many current websites.

But World of Warcraft will never run in Firefox (not at least without a severe redefinition of "browser", "runtime", or "OS").

What is a platform?

What might the average computer look like in 10 years from the perspective of application platforms?

I feel that each computer will consist of several rings or layers of runtime platform:
  1. Native: applications are programmed against the native OS libraries, and constrained by native OS security model (ex: win32, POSIX, OpenGL, etc.)
  2. Managed: applications are programmed against a large, general purpose, coherent, high-level API; and run in a virtual machine with detailed, explicit, and programmable security model (.NET, Java, Python, etc.)
  3. Internet: applications are programmed against a special-purpose API, with clearly defined semantics, which is exposed though a common data model (the DOM for web) and standard set of operating primitives (HTTP for the web); security model is highly sandboxed, limiting access to private data where not explicitly required by the data model or operating primitives (IE, Firefox, and one day Virtual World Viewers?)
I expect that in the future, each computer will come equipped with one OS, a handful of managed runtimes, and precisely two internet application platforms: a web browser and a virtual world viewer.

How would the web look in a VW viewer?

I think it would be wrong to assume that because 2D embeds in 3D mathematically, that the web browser should go away -- entirely subsumed in a 3D viewer. That really ignores the long and ongoing history of humans quite preferring to do a lot of their most abstract work in 2D documents. It would also really undermine my own argument that the web and VWs are really very different modes of interaction.

Instead, I believe that the VW viewer will know of the system's web browser, and rely on it directly for all web-related tasks. Examples might be embedding a browser window for the purpose of authenticating with an OpenID or oAuth provider for some network resource, such as gaining access to your facebook profile, or to send emails through gmail. Web applications like Google Apps would also make a simple and effective document sharing service available from within a virtual space.

So from a user's perspective, the web browser would be rendering elements of the regular user interface, or displaying discrete web pages in an integrated manner without appearing as a full web browser. There will also be cases of rendering to texture in order to display web pages as in-world objects, but their purpose will be peripheral.

As a some what related aside, I find it constantly amusing to see VW implementers try to embed 2D documents in 3D spaces. With few exceptions, there is no benefit whatsoever to being able to see your spread sheet in perspective projection as a texture. Give me a use case where having 2D documents in 3D is more than a gimmick, and I'll give you a modified use case where dimensionality is irrelevant, and the interaction feels much more natural.

Not just the browser

Other internet applications that I feel should not be subsumed within the virtual world viewer, but instead should integrate smoothly with them using an inter-process communication channel are things like E-mail, IM/Chat networks, Streaming Media, Office programs, etc.; all of which are regular desktop applications that the viewer calls out to in response to in-world stimulus; or alternatively, will respond in-world to a stimulus from the user's desktop.

Imagine a "meet in-world" button on your IM client, or having the streaming audio in your virtual home synced with your current desktop play-list (or vice versa).


internet 3d said...

I tend to believe that browsers will so much change in the next 10 years that they will easily be able to portray 2D and 3D internet.

Thanks for this interesting essay.

Stephen W said...

I know this is a little off topic as to weather a virtual world should be delivered via a web browser or be stand alone, but it does touch on a topic that I see frequently.

Why would you want to browse the web as a virtual world? I can think of some specific applications that would be great, but the whole web as a virtual world sounds very restrictive to me.

My view is that your making a common mistake in the assumption that we are going to ONLY want to browse the web as a virtual world where you can see all the other people browsing and chatting etc.

If this was a good thing then all sites would have an integrated chat systems right now. The fact that they don't illustrates that for the majority of sites out there, this is not appropriate.

Therefore I belong to 3rd 'school of thought' that think the requirements for a 3D Web / internet start at a much simpler level providing the means to immerse and interact with content in a natural 3D way.

This allows for all manor of uses as rich and varied as the web currently is that include applications & services such as Virtual Worlds etc.

The difference is the better integration that would be possible as you browse between these different uses.

It will be some time before this is fully or even partially realised & some standards for usability etc. would need to emerge, and yes I agree a new browser would be required to adapt to the new way we interact with the web.

However, until we reach this point though, weather its delivered through a web browser or stand alone application, doesn't really matter, there's pros and cons with both.

I feel this common assumption as a whole is hindering progress for 3D Web. For example, all the companies that flocked to use second life in this way to eventually realise (not all have even now) that really its not the 3D Internet at all, and was never conceived to be.

More information on our approach:


R. McDougall said...

Thank you for the counter point Stephen. However I think I may have not been clear.

I think my intention was to state that both web and VWs are different experiences and therefore have different browsers; that one doesn't go in the other.

Except that for limited use cases, it makes sense to put 3D in a web browser, like Google's O3D, or put web in a VW browser, like HTML-on-textures.