TechYob

Went along to an interesting panel discussion about the future of TV and the internet hosted by AIMIA.  My take is that P2P will be to content as broadcast was to TV.  When the medium is the internet, P2P is hands down the most efficient and scalable way of using ’spectrum’ (bandwidth).  Of all the speakers I found myself nodding more often than not when Phil Morle waxed lyrical on the subject. 

Sydney is a small eco system in the startup world and got talking with Marty of Tangler fame about the importance of instituting a customer development cycle as part of the ethos of every Oz tech startup.  The basic concept is that you dont scale your sales and marketing before you have established a repeatable sales process.  To do this you need to have a rigorous set of gates, just as you would for developing your product, which you use to filter your business idea or product concept. 

One last mention goes out to Rob and Nick who are developing a remarkable sports oriented online channel 3eep.  Sound plan, great team.  These are the kind of guys that News and the TV channels need to be getting behind otherwise they will find the train will have already left the station when their panopoly over free to air TV inevitably looses steam. 

Interesting approach to folksonomy and the semantic web called Freebase.  Im a big believer in a bottom up approach in comparison to building a structured ontology (see the Cycorp project).   However, in order to be a truly useful service it needs some form of antisubverion built in, otherwise it just becomes another platform for spam.  Therefore ultimately there will need to be a combination of algo and human-tagging for the semantic web to become truly useful.  Last year I built an alpha product with a similar approach to Freebase, but also overlaped social networks as a way of building relevance and subversion resistance.  One of the biggest challenges they will find is being able to scale out to a respectable number of users, as internetworking graphs of meta-data presents a very real indexing and real-time query challenge.  I look forward to seeing more projects in this space.

An honour to meet Vint Cerf, co-founder of the Internet Protocol, at the University of Sydney today. A humble, eclectic genius with a sense of humour.

To paraphrase Kahil Gibran, all genius is obvious in hindsight. The fact that 1 billion people (16.6% of the worlds population) benefit from his ingenuity without a second thought is testimony to the scale of his contribution to humanity. Nuff said.

I gave the Google MapReduce paper a quick read today. It outlines the approach Google uses to abstract over paralleled computation, distribution of data and fault tolerance associated with deploying large data-sets over large clusters of commoditised hardware. Bottom line, any web startup with significant scale aspirations should raise MapReduce or the open source Hadoop as a brown-bag topic for the engineering team.

Thoughts:

1. Adapting a Functional programming model and the discipline of atomic task execution is a smart way to reduce complexity for many internet scale information processing problems.
2. The MapReduce library significantly reduces the knowledge acquisition phase for new Google engineers, and hence helps scale people as much as it helps to scale code.
3. MapReduce is already being used in some machine-learning applications. It is likely that the corollary is also true. Numenta’s hierarchical temporal memory (HTM) model is very interesting in its addition of cascasding expectation optimization.
4. Prioritization of tasks corresponding to different business applications would be an emerging issue. If the end-game is building an internet operating system to be made available to all developers a la Amazon’s Elastic Computing Cloud (and it should be!) this resource management and task isolation is key.
5. Could the master-worker communication be replaced with P2P between workers?
   

Barcampsydney is here to stay - an un-conference dedicated to exchanging ideas and knowledge around all things geekery.  Fantastic effort by Mick Liubinskas, Christy Dena, Jason Yip, Rich Buggy and Russ Weakley.  Stand out presentations for me were ex-Microsoft Joel Pobar’s discussion of google-style scale out programming models, which got me reading the MapReduce paper over breakfast this morning, and Marty and Mike’s boot-strapping-a-business presentation.  As ever, it was great to catch-up with Phil Morle and hear the interesting work he is doing with Yoick and P2P virtual worlds. Welcome also goes to Mike Perrow who has just moved to Oz to start with Google. 

The event was free, and I got a free t-shirt, so an acknowledgement to the sponsors: 

My only suggestion is that next year there actually be a bar to camp at.