Posted by Anil Madhavapeddy
Sat, 01 Nov 2008 16:17:00 GMT
My favourite session of PDC 2008 was by Anders Hejlsberg as he described the Future of C#. Anders is an excellent speaker, and it was educational watching how he made fairly complex type systems come across to an audience more used to simpler programming languages. I say this after attending ICFP recently, where that wasn't true of all the presentations!
The high-level message was exactly what I wanted to hear: in order to exploit multi-core processors more effectively, Microsoft's languages are focussing on improvements in three areas:
Declarative constructs: making the programmer define overall goals and constraints enables a tool-chain to more effectively scale computation across multiple processors. An example of this in C# 3.0 is the Language Integrated Query (LINQ) domain specific language for defining queries.
Dynamic interoperability: Although Anders is a big fan of static type systems (robust, high-performance, enable intelligent tools), he pointed out that a lot of the systems that C# programmers have to deal with are fundamentally dynamic. Examples are web interfaces such as SOAP/REST APIs, COM objects for talking to other Microsoft applications, or scripting language integration (e.g. Python or Ruby).
Concurrent programming: dealing with multiple cores requires splitting up computation into parallel threads, and the resulting parallel mass of code is often hard to debug (e.g. difficulties of reproducing obscure concurrency issues) and statically reason about.
An interesting aside was his assertion about "co-evolution". Microsoft have a number of languages which have recently been unified under the CLR, such as Visual Basic, C# and most recently F#. Rather than have one language race ahead of the others, they like to "borrow" features as appropriate into other languages. This is obviously made easier by having a common run-time foundation, and an example of co-evolution is the introduction of lambdas into C# 3.0 as obviously seen in F#.
Concurrency
Anders observes that all the attempts to automatically parallelize software has not yielded good results in the past, and so support needs to be built into the language to do "top-down" parallelism instead of the compiler inferring it from the bottom-up. To this end, they are introducing parallel extensions into the .NET framework. This makes use of functional features to parallelize code, including LINQ queries or CPU-intensive compute.
For an example of how this code looks, check out Jurgen Van Gael's post on using F# with the Task Parallel Library. I wanted to learn more about this, and wandered over to the Hands On Labs where F# and Visual Studio 10 were all pre-installed. I then ran into an extremely cool feature of F#... asynchronous workflows.
F# introduces an extension to the usual let ML operator in the form of let!. As Don Syme explains on his blog ), this can be interpreted as "run the asynchronous computation on the right and wait for its result. If necessary suspend the rest of the workflow as a callback awaiting some system event". So this construct lets you write straight-line code which can potentially block, without the hassle of spawning threads or encoding continuation passing style constructs in the code. Already a nice improvement over OCaml! (although to be fair I've not had a chance to check out some of the parallel OCaml extensions).
Dynamic Programming
Dynamic programming is the main focus of C# 4.0. In order to support dynamic objects as first-class citizens in a statically typed world, they introduce a dynamic static type. This forces all method resolution on that object to happen at run-time and disables static checks by the compiler (aside from looking from mixups between dynamic and static objects). The dynamic type now makes it easy to wrap support for Python, Ruby and Javascript, since the relevant dispatch functions can all be hidden away in the method resolver at run-time, leaving the programmer with a single syntax for invoking methods across these different languages.
The actual definition of method resolvers is pretty straight-forward; he demonstrated custom getters and setters (similar to Python for example) by using the IDynamicObject interface to define actions to take when properties are accessed. His example did the usual dictionary wrapper which mapped setting arbitrary properties onto an internal dictionary variable.
Another improvement in this space is the addition of optional arguments and labelled arguments. Both of these have well-defined semantics (optional arguments have to come after non-optional ones, and evaluation of arguments is left-to-right) and are purely syntactic improvements with no run-time cost. One of the best examples he showed of using these around COM interoperability. In current versions of C#, due to the lack of named arguments a common function such as "Save As" might require 12 or more stub arguments to be specified as ref missing. Now, those long, repetitive lines can be folded down to only the arguments which are required.
Sort of related to the earlier co-evolution was his demonstration of how similar Javascript and C# syntax is now with this new support. He took a Silverlight 2 code snippet written in Javascript and ported it over to C# 4.0 with some very mechanical changes. The languages are definitely converging fast!
If you're interested in checking out more on this topic, look at IronPython, the Dynamic Language Runtime (DLR), and the TL10 session on dynamic languages on .NET. The DLR is currently maintained as part of IronPython, but is being moved out to sit on top of the CLR for more languages such as Visual Basic, Javascript or COM. The DLR covers common optimizations which are useful for scripting languages, such as call-site caching, dynamic dispatch and expression trees (e.g. for LINQ). The DLR has a bunch of binders which bridge between different backends such as .NET, Silverlight, the native Python or Ruby backends, or COM applications such as Microsoft office.
Meta-programming
Later on at the Future of Programming Languages panel session, Anders talked about meta-programming as being one of the future improvement he's looking at. Currently, there is a lot of ad-hoc code generation in place when creating Windows applications, and unifying this into the language would give safety and maintainability improvements.
In order to do this, for C# 5.0 they are rewriting the compiler to be self-hosting in C#, since it has historically been a C++ application. This permits them to switch the compiler from being a traditional "black box" compiler to a hosted .NET service which can be called directly by .NET programs in order to do dynamic run-time compilation of code. Other portions of the compiler chain are also exposed to permit incremental program construction by third-party code.
He demonstrated this with a pretty nifty C# top-level, into which he directly typed Winforms code to construct a window with a few simple buttons using the C# compiler server. Not to be outdone by this, Miguel de Icaza promptly upstaged Anders at his (fantastic) Mono 2.2 session. He demonstrated the new C# shell which is present in Mono trunk builds and can essentially be used like an OCaml or Python top-level to mess around and manipulate C# code. He also talked about embedded Mono and SIMD support which pushes their compiler ahead of Microsoft's in the 3D performance game.
Summary
I'm firmly convinced about the potential of F# now. I had the opportunity at the Open Spaces area to quiz Scott Guthrie about whether or not F# was a toy language. He replied using the same arguments as Anders that the higher-level language approach (declarative, functional) was very important strategically to Microsoft to let their developer platform continue to survive in a multi-core world.
This boils down to the individual languages not being that important any more (as seen by the sharing of features between C# and F#), and the underlying execution layer (the CLR/DLR) adding efficient support. Now any old language can adopt higher-level features without having to re-do all the optimization grunt work again and again. Much like Xen offers a new golden age for innovative new OS research by freeing programmers from writing a million hardware device drivers, it looks like .NET is ushering in a new age of programming language innovation!
Inspired by the PDC talks, I've got MonoDevelop and F# up and running on my Macbook Air, and am just playing with GTK# and CocoaSharp#. If this works as well as OCaml, then it might finally be time to abandon the old stalwart and move to a new language for my day-to-day stuff!
Posted in research, hacking | Tags pdc2008 | no comments
Posted by Anil Madhavapeddy
Tue, 28 Oct 2008 15:48:00 GMT
I'm over at the Microsoft PDC in Los Angeles this week, and commuting over from Anand's place in Camarillo using the Metrolink. The 3 hour commute gives me a great excuse to catch up on my TED talks on my iPhone.
The entire set of talks is available online via iTunes, and these really stood out:
Jared Diamond on Why Societies Collapse : he talks about lots of civilizations which have reached tipping points and have collapsed, often catastrophically. Not a funny or entertaining one, but lots of solid, well-researched facts. Also see his book which George Dunlap recommended to me but is still in my reading list.
Mihály Csíkszentmihályi on Creativity, fulfillment and flow : again, a really well-researched talk which categorizes human motivation into a 2D space of challenge vs skill, and the different emotions which are evoked by being in different situations (e.g. apathy requires no challenge and no skill, and anxiety results if you have low skill and a high challenge ahead of you). The "flow" he argues for is the ideal pinnacle of where we should aim to get to in our lives.
Finally, what I consider to be the greatest TED talk I've ever seen is by the late Paul MacCready on Nature vs Humans, and what we can do about it : he starts off slowly by showing some graphs about the impact of humanity on the population of the planet (hint: we've taken over), and finishes off with some truly awesome demonstrations of natural planes, such as Ornithopters or solar-powered tiny fliers that do not require gas to fly. Highly recommended if you only watch one TED video at all!
There are about 80 videos left on my viewing list.. this is going to take a while!
Posted in research, art, press | no comments
Posted by Anil Madhavapeddy
Sun, 01 Apr 2007 22:12:00 GMT
It's been a busy old March (release management of the new XenEnterprise sucked up most of it). I did take a break and go over to EuroSys 2007 in Portugal to present the language and compiler I implemented as part of my PhD work (read it here).
Although the talk I gave was was a bit underwhelming (more preparation and time-practise next time!), I met a whole bunch of really interesting people. My argument about rewriting whole applications also didn't get laughed out the room as I thought it might, as people recognise that retro-fitting safety enhancements on existing languages is a bit of a dead-end road to go down. It has definitely inspired me to make more time to spend on polishing up the Melange applications for a proper release in 2007.
In a pleasant surprise, it also won the Best Student Paper award of the conference as well!
Posted in research | 2 comments
Posted by Anil Madhavapeddy
Sat, 30 Dec 2006 01:11:00 GMT
Inspired by finishing my PhD corrections (!) today, I decided to hook up the DNS server from our Melange project up to the Internet. The authoritative server is called deens (since the co-author is one Tim Deegan, geddit?), and is written in pure OCaml.
This is all rather experimental, to put it mildly, but I stuck in the zone file below, hooked it up as a delegate to our main name-servers, checked it against the DNS Report, and it all seems to be working!
$ORIGIN deens.recoil.org. ;
$TTL 240
deens.recoil.org. 604800 IN SOA (
deens.recoil.org. anil.recoil.org.
2006122401 3600 1800 3024000 1800
)
IN NS ns1.deens.recoil.org.
IN NS deensns.recoil.org.
ns1 IN A 194.70.3.132
dynamic IN CNAME dynamic.recoil.org.
static IN CNAME static.recoil.org.
anil IN CNAME dynamic
stats IN CNAME dynamic
I also modified stats.recoil.org to be an alias to stats.deens.recoil.org, so all the requests for that domain will go via the deens setup. You actually need a user/pass to access the site, but that doesn't matter; if it gets that far, the DNS bit has worked.
There's still an awful lot of tedious work to get the server into a production-ready state, such as proper logging, more error handling and recovery, etc., but I really hope to find the time in 2007 to polish this up somewhat. Performance is excellent already; faster than BIND by quite a lot, and it can optionally use more memory to cache responses to shoot up to crazy levels.
Incidentally, the dig replacement utility also seems to be working fairly well, and David Scott has been messing around with a Bonjour implementation that will get finished sometime in 2007 as well (honest!).
Posted in research, hacking, net | no comments
Posted by Anil Madhavapeddy
Wed, 30 Aug 2006 21:35:00 GMT
An oft-cited criticism of virtualisation is that 3D hardware acceleration doesn't work, preventing you from enjoying your hard-earnt game of Quake 3. Rumours abound that Parallels is developing it for its software, and that VMware is doing something in this area as well.
However, thanks to the Google SoC, the power of open-source itching, and the talented Andrés Lagar-Cavilla, Xen now has support for 3D acceleration as well! Check out the xen-gl web-page with screenshots, or just clone xen-gl.hg and get hacking!
Rather than getting down and dirty with foreign grant mappings, PCI pass-through and all that malarky, Andres adopted for the more pragmatic approach of packetising OpenGL using the Chromium project, and creating an x.org module to correctly position the resulting OpenGL. End result: hardware rendering in a guest domain, without requiring any extra hardware privileges. Awesome to the max!
Posted in research, xen | no comments | no trackbacks
Posted by Anil Madhavapeddy
Mon, 10 Jul 2006 11:43:40 GMT
After an invigorating 120 minutes of questioning, bright lights shining in my face, and general all-round cross-examination, my two examiners Ian Wakeman and Tim Griffin decided to pass me with minor corrections for my PhD! On top of that, it miraculously got nominated for a BCS Distinguished Dissertation award, which does motivate me to really polish it up before submitting the final version. At the Volta Lounge, we are planning to graduate together next summer, so there's no rush...
I haven't quite gotten used to it yet; while booking tickets to San Francisco for Thursday I bottled out to the sales lady and put my title down as "Mr." instead of "Dr."!
Posted in cambridge, research | no comments | no trackbacks
Posted by avsm
Tue, 08 Nov 2005 19:25:22 GMT
I've been majorly focused on finishing off my PhD Thesis recently, hence the lack of updates (but check out the sharp green gradient I'm posting on the thesisometer!).
While researching the history of dynamic bounds checking in languages, I found this remarkable quote from Sir Tony Hoare in his 1980 Turing Award lecture about Algol-60:
A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interest of efficiency on production runs. Unanimously, they urged us not to - they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980, language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law.
Bear in mind he made this statement in 1980, and nothing has really changed in the intervening 25 years as the Internet gets overrun by viruses and worms which take down hospitals and nuclear power plants.
Another amusing one is about Fortran, found on his Wikiquote page:
On October 11, 1963, my suggestion was to pass on a request of our customers to relax the ALGOL 60 rule of compulsory declaration of variable names and adopt some reasonable default convention such as that of FORTRAN. [...] The story of the Mariner space rocket to Venus, lost because of the lack of compulsory declarations in FORTRAN, was not to be published until later."
Posted in research | no comments
Posted by avsm
Fri, 16 Sep 2005 14:33:43 GMT
One of the more interesting discussions I had during Ubicomp was with Tim Kindberg and a very nice chap from France Telecom who lived in Japan for a few years. He was very familiar with the state of barcode tagging as deployed in Japan, and these points stood out:
Most barcodes in use here are QRCodes and NTT phones come pre-installed with a reader. Users can also install a UPC barcode reader. The first such use of QRCodes you see is in your passport; the entry stamp has a barcode on the sticker with your id number on it.
QRcodes are so easy to read since most camera-phones in Japan are auto-focus, in contrast to our crappy fixed-focus attempts here. Our tests with QRcode reading using the KDDI cellphones were pretty successful due to that alone.
QRCode deployment in Japan is by no means ubiquitous, as reports in some blogs suggest. You find them on some products (like, oddly enough, tissue packets) but most advertising posters are distinctly QRCode-free. I certainly never saw people clicking on them in public over around 3 weeks of wandering around the country.
The telcos here draw a clear distinction between content providers and application providers. Phones aren't quite as programmable as in the West, and so most barcodes take you to a webpage portal with various actions (such as buying the product or just linking it for future reference). Of course, phones in Japan have cheap high-bandwidth connectivity, so this works very well without the long latencies and download times that we have to put up with on our carriers. We confirmed this by playing with the phones KDDI provided at the conference.
One really interesting example was that mobile phone bills sent out to people in the post have a QRCode on them, which, when clicked, is stored in one of the phone "barcode slots" in the reader application. Users then go to a post office or bank, and can pay that bill by pressing their phone against an RFID reader (which pays the bills for all the QRCodes stored on the phone). A superb example of a pick and drop interface in the wild.
Posted in research, ubicomp | 2 comments
