Ethereum: Is it cost effective to create reliable contracts?

March 15th, 2015 No comments

The idea of embedding of a virtual machine supporting user executable code inside a cryptocurrency continues to fascinate me. I attended the Ethereum workshop on using Solidity to develop dapps (distributed apps) yesterday.

Solidity seems to have been settled on as the high-level language in which programs for the EVM (Ethereum Virtual Machine) will be written, at least at launch. While Ethereum appear to know what they are doing in the design of the cryptocurrency (at least to my non-expert eyes), they are rank amateurs at language design. A good place to start learning is the rationales for Ada, Eiffel and Frink for starters.

Ethereum uses the term Contracts to describe programs executed by the EVM.

Anybody publishing a Contract that involves transferring something that has a real-world impact on their financial state (e.g., funds in cryptocurrency that are exchangeable for government backed currency) will want a high degree of confidence in the reliability of the code.

It is easy to imagine that people will be actively reverse engineering all published Contracts looking for flaws that can be exploited to siphon off funds into their personal accounts.

Writing high reliability code is time consuming and expensive. One way of reducing time/cost is to use a language that implicitly performs lots of checking, rather than requiring the developer to insert lots of explicit checks.

The following is an example of a variable definition providing information to the compiler that can be used to insert checks in the generated code (e.g., that no value outside the range 1 to 1000 is assigned to amount_to_pay); the developer does to not need to worry about explicitly adding checks in all the necessary places. This language functionality was invented in 1970 (in Pascal) but went out of fashion, for new languages, in the 1990s.

var amount_to_pay : 1..1000;

Languages such as Ada and Frink improve program reliability by providing functionality that reduces the effort developers need to invest in checking for unintended behavior.

Whatever language the code is written in, what happens if an attempt is made to assign 1001 to amount_to_pay? It does not matter whether the check is implicit or explicit, an error has occurred and has to be handled. As a developer of this code I want to know about the error (so I can fix it) and as a user of the code I do not want to lose money for executing a failed transaction.

At the moment the only Ethereum error handling solution I can think of is writing error information to the blockchain (in place of the information that would have been written on successful program execution); the user will still get charged for executing the program but at least will now have the evidence needed to obtain a refund (the user has to be charged to prevent denial of service attacks using Contracts containing a known fault).

I wonder who will find it worthwhile investing in creating the high reliability code needed for Contracts to be a viable solution to a problem?

Been invaded by mysterious sounds? You need SoundHound

March 12th, 2015 No comments

Team SoundHound (Gary, Pavel and yours truly) were at the Intel Hardware Hackathon last weekend. Hardware hacks are about the peripherals that are available on the day; I grabbed one of everything and we all sat down to play with them and come up with a great gadget to build. At an Intel sponsored event the computers are obviously Galileo and Edison (which is what we ended up using).

The Intel guys had a very fixed idea about how the hackathon was to run and specified that eight projects had to be proposed and selected (by everybody present) on the Friday night and that these would be worked on over the weekend (but we were allowed to change the idea; the logic escaped me). I proposed a vibration detection project to measure the impact of passing vehicles on houses next to busy roads. Nobody on what was to become SoundHound thought this was a great idea, but it was the only idea we had and there was nothing stopping us radically changing our mind over the weekend.

Vehicles generate sound which we can hear, but buildings are damaged by the lower frequency vibrations that we sometimes feel but don’t usually hear. A microphone would be useless, what was needed was a vibration sensor and a packet of piezo vibration sensors, the SEN-09198, was provided. We did not have any amplifiers to boost the signal, so the sensors were wired directly onto the signal pins; this meant that the dynamic range of the signal was about 50 times smaller than the input could handle and our test vibrations were created by banging on the table on which the sensor sat (power spectrum of me doing just that below).

Power spectrum of fist banging on table

We worked away on the vehicle vibration idea waiting for something better to come along. Some household appliances contain motors that cause them to vibrate, perhaps we could detect these vibrations. The use case here was a fridge next to a wall transmitting vibration into another room which then generated a hard to pin-down noise in that room. Around this time it occurred to us that a ghost detector (everybody knows that walls vibrate when ghosts travel through them) might be a bigger market, after all occult books sell well.

IBM were an event sponsor and had people on-hand to help us set up and use Bluemix, their cloud offering. The Edison board was communicating with our laptops via wifi and this got rerouted so the data was sent to IBM’s cloud storage. Collecting everybodies vibration data meant we could do pattern matching on the power spectrum of the vibration to identify candidates for the device most likely to be generating it (whether the vibration characteristics of different devices is sufficiently different for the pattern matching to be able to distinguish them at some useful level is another matter; I imagine that the shape of a fridge causes its vibration to modulate slightly from the frequency of the local electricity supply).

One of the problems of producing a working demo at a hackathon is being held to a much higher standard of reality. Teams whose output is slideware illustrating the tenuous connection its members have to reality don’t seem to have any trouble getting unquestioning agreement that anything they propose will just work. We were asked how we would obtain the vibration data for different household devices, against which the customer data could be compared; a marketing type capable of convincingly talking about our crowd source solution would have been useful.

Late Saturday afternoon saw team SoundHound out on the streets measuring vehicle vibration. Unfortunately large lorries traveling at speed are thin on the ground in central London, also guys standing next to a laptop with wires attached to something on the ground filming everything that went past had the effect of causing drivers to slow down to low vibration speeds (rather than speeding up to get away from us).

Perhaps we should have gone with the ghost detector sale pitch. The cloud data storage could have been rebranded as The Ethereum data matrix and flashing leds added to the hand-held sensor (the penguins below send the wrong message).

If it is possible to distinguish home appliances based on their vibration characteristics, and a database is created, there is an opportunity for salesmen to systematically check each house/flat ‘listening’ for appliances that are close to failing (motors vibrate more as they wear out): “Hello Sir, your fridge/boiler/air-conditioning is getting old and this week we have a special offer on replacements…”.

The picture below shows our 3-axis vibration sensor (plastic mounting/vibration interface laser cut by Elen from the FabLab, who do brown bag lunch and learn sessions if you are ever in central London). Two axis would probably been enough, and maybe even one, but we had the sensors and wanted to impress.

3-axis vibration sensor

Extracting the original data from a heatmap image

March 4th, 2015 No comments

The paper Analysis of the Linux Kernel Evolution Using Code Clone Coverage analysed 136 versions of Linux (from 1.0 to and calculated the amount of source code that was shared, going forward, between each pair of these versions. When I saw the heatmap at the end of the paper (see below) I knew it had to appear in my book. The paper was published in 2007 and I knew from experience that the probability of seven year old data still being available was small, but looked so interesting I had to try. I emailed the authors (Simone Livieri, Yoshiki Higo, Makoto Matsushita and Katsuro Inoue) and received a reply from Makoto Matsushita saying that he had searched for the data and had been able to find the original images created for the paper, which he kindly sent me.

Shared code between Linux releases

I was confident that I could reverse engineer the original values from the png image and that is what I have just done (I have previously reverse engineered the points in a pdf plot by interpreting the pdf commands to figure out relative page locations).

The idea I had was to find the x/y coordinates of the edge of the staircase running from top left to bottom right. Those black lines appear to complicate things, but the RGB representation of black follows the same pattern as white, i.e., all three components are equal (0 for black and 1 for white). All I had to do was locate the first pixel containing an RGB value whose three components had at least one different value, which proved to be remarkably easy to do using R’s vector operations.

After reducing duplicate sequences to a single item I now had the x/y coordinates of the colored rectangle for each version pair; extracting an RGB value for each pair of Linux releases was one R statement. Now I needed to map RGB values to the zero to one scale denoting the amount of shared Linux source. The color scale under the heatmap contains the information needed and with some trial and error I isolated a vector of RGB pixels from this scale. Passing the offset of each RGB value on this scale to mapvalues (in the plyr package) converted the extracted RGB values.

The extracted array has 130 rows/columns, which means information on 5 versions has been lost (no history is given for the last version). At the moment I am not too bothered, most of the data has been extracted.

Here is the result of calling the R function readPNG (from the png package) to read the original file, mapping the created array of RGB values to amount of Linux source in each version pair and calling the function image to display this array (I have gone for maximum color impact; the code has no for loops):

Heatmap of extracted data

The original varied the width of the staircase, perhaps by some measure of the amount of source code. I have not done that.

Its suspicious that the letter A is not visible in some form. Its embedded in the original data and I would have expected a couple of hits on that black outline.

The above overview has not bored the reader with my stupidities that occurred along the way.

If you improve the code to handle other heatmap data extraction problems, please share the code.

Hardware variability may be greater than algorithmic improvement

February 24th, 2015 No comments

I’m giving a talk at COW 39 this week and it is more user friendly to include links in this summary than link to a pdf of the slides (which actually looks horrible).

Microelectronic fabrication has now reached the point where it etches and deposits handfuls of atoms (around 20 or so). One consequence of working with such a small number of atoms is that variations in the fabrication process (e.g., plus or minus a few atoms here and there) can have a significant impact on component characteristics, e.g., a transistor consumes more/less power or can be switched faster/slower. It might be argued that things will average out over the few hundred+ million components inside a device and that all devices will be essentially the same; measurements show that in practice there is a lot of variation across the devices.

Short version: Some properties of supposedly identical microelectronic devices now vary by around 10% and this variability is likely to get larger in the future.

Nearly all published papers involving computer system power measurement are based on measuring a single system. Many of the claimed algorithmic improvements are less than 10%, i.e., less than the expected variation in power consumption across supposedly identical devices/systems. These days any empirical paper involving power consumption has to include measurements from many devices/systems if any credibility is to be given to the findings.

The following measurement data has been found while researching for a book (code and data used for talk); a beta version of the pdf will be available for download real soon now.

The following plot shows feature size, in Silicon atoms, of released microprocessors. Data from Danowitz et al.

Line width of latest devices over time

A study by Wanner, Apte, Balani, Gupta and Srivastava measured the power consumed by 10 separate Amtel SAM3U microcontrollers at various temperatures (embedded processors get used in a wide range of environments). The following plot shows how power consumption varied between processors at different temperatures when idling and executing. The relationship between the power characteristics of different processors changes with temperature; a large amount when idling and a little bit when executing.

Power consumption at various temperatures

A study by Balaji, McCullough, Gupta and Agarwal measured the power consumption of six different Intel Core i5-540M processors, running at various frequencies, executing the SPEC2000 benchmark. The lines in the following plot were fitted to the data (grey crosses) using linear regression. The relationship between the power characteristics of different processors changes with clock frequency.

Power consumption at various frequencies

A study by Bircher measured power consumption of various system components, of an Intel server containing four Pentium 4 Xeons, when executing the SPEC CPU2006 benchmark. The power distribution for mobile devices would show the screen being the largest user of power, but I don’t have that data).

Power consumption at various frequencies

A study by Bircher measured memory (blue) and cpu (red) power consumption, in watts, executing the SPEC CPU2006 benchmarks. No point optimizig cpu power if over half the total power is consumed by memory.

Memory and DIMM power consumption for various programs

A study by Krevat, Tucek and Ganger measured the performance of then modern disk drives originally sold in 2002 (left) and 2006 (right). Different colors showing through on the right indicates that some discs have different performance characteristics than the others (faster performance -> less time -> less power, i.e., I don’t have power data for disk differences)

I/O performance of disc drives sold in 2002 and 2006

A study by Kalibera et al executed a benchmark 2,048 times followed by system reboot, repeated 10 times. Performance is consistent within a particular reboot, but not across them.

FFT benchmark performance after reboots

The following plot is in the slide deck and is included here as a teaser (i.e., more later). It shows data for 2,386 processors (x-axis is slowdown, y-axis clock frequency, top right legends max power), which thanks to Barry is now publicly available (or at least will be once some web details are sorted out).

Line width of latest devices over time

Long tail licensing

February 16th, 2015 No comments

Team ‘Long Tail Licensing’ (Richard, Pavel, Gary and yours truly) took part in the Fintech startupbootcamp hackathon at the weekend.

As the team name suggests the plan was to implement a system of payment and licensing for products in the long tail, i.e., a large number of low value products. Paypal is good for long tail payment but does not provide a way for third parties to verify that a transaction has occurred (in fact Paypal does its best to keep transactions secret from everybody except those directly involved).

Our example use case was licensing of individual Github repositories. Most of today’s 3.4 million developers with accounts on Github would rather add more features to their code than try to sell it; the 16.7 million repositories definitely qualifies as a long tail of low value products (i.e., under £100). Yes, Paypal could be (and is) used as a method of obtaining payment, but there is no friction-free method for handling licensing (e.g., providing proof of licensing to third parties).

Long Tail Licensing’s implementation used cryptocurrency for both payment and proof of licensing (by storing license information in the blockchain). For the hackathon we set up out own private Bitcoin blockchain to act as a test rig, supply fast mining and provide near instantaneous response.

To use Long Term Licensing a developer creates the file .cryptolicense in the top level directory of their repo; this file contain information on the amount to pay, cryptocurrency account details and text of licensing terms. A link in the file points at our server, which validates the .cryptocurrency file and sets up a payment transaction from the licensee’s Bitcoin wallet; the licensee confirms the transaction and the payment is made.

The developer’s chosen license information is included in the transactions blockchain, providing the paperwork that third-parties can view to verify what has been licensed. This licensing information could be in plain text or use public key encryption to restrict who can read it (e.g., eBay could publish a public key that third parties could encrypt information so that only eBay’s compliance department could read it).

The implementation code includes links to private servers and other stuff that it should not be be; hackathon code is rarely written with security in mind. So those involved would rather it not be pushed to Github (perhaps it will get tidied up and made suitable for public consumption at a later date).

We did not win any of the prizes :-( . Well done to Manoj (a frequent hackathon collaborator) and his team for winning the $100k of Google cloud time prize.

Do languages evolve to minimise portability?

February 10th, 2015 No comments

Governments promote standards because following them helps their citizens save money. The UK and US have contrasting positions, with the UK focusing on savings achieved through the repeated use of standardized items and the US focusing on the repeated use of skills people acquired through using a standardized item.

What benefit, if any, do product vendors receive from adhering to a standard? A few vendors may be in a position to use economies of scale to gain market share (by driving out of business less well funded competitors), other vendors follow a specified standard as a means of winning a contract with large customers, and the rest don’t care.

In the program language world, for 20 odd years, there were just the Cobol and Fortran standards. These started out idealistically driven because hardware vendors had to convince potential customers that it was commercially viable for them to write software for the products they were selling (i.e., the early computers). Once business started to take off the computer industry’s compete/cooperate dynamic, mixed with large egos wanting their own way and the sales driven mandate to keep existing customers happy (i.e., don’t break existing code) took over. There was lots of infighting and progress was slow, but at least there were lots of people who thought it important enough to be involved.

In the mid-1980s there was a seismic shift in approach to language standards with Pascal and C becoming ISO standards, the first created by a group of mainly academics and the second by a group containing many small company consultants/vendors. The age of ‘activist’ language ISO standard creation had begun; which is not to say that anything changed in the Cobol and Fortran standards’ world. For a look back at this decade see: Computer Standards & Interfaces, volume 16, numbers 5 & 6, Sep 1994.

If you were to put somebody who knew nothing about computer languages in a room with the standard’s committees for Modula-2 and C++, I don’t think they would be able to tell them apart. Inside the bubble its not possible to distinguish the language that died before its standard was published and the language that continues to grow like Topsy.

Activists want to get things done and will select the route of minimal bureaucratic resistance. These days this means that new language standards are often created locally and fed into the ISO process higher once they are done.

Open source compilers have solved many of the source portability problems caused by language dialects, that standards were intended to solve, by dramatically shrinking the market of the compilers that supported those dialects.

Where has the user been in all this? Well, where-ever they are, they are rarely seen at language standard meetings.

These days portability problems are caused by the multitude of languages, not by the multitude of dialects of a language.

Top secret free information

January 26th, 2015 No comments

I may or may not have been at a top secret hackathon at the weekend.

Secret hackathon wording

If you read the above text please leave your contact details in the comments so I can arrange to have somebody come round and shoot you.

In other news, I saw last week that the European Delegated Act requires that the European Space Agency to provide “Free and open access to Sentinel satellite data…” A requirement that is not incompatible with the wording above.


Live coding, where should I send my invoice for performance royalties?

January 19th, 2015 4 comments

Live coding (i.e., performance art + coding) is the shiny new thing in trendy circles (which is why it was a while before I heard about it). I’m sure readers have had the experience of having to code around issues on customer sites while everyone looked on, this is certainly a performance and there is an art to making it look like minor issues, rather than major problems, are being sorted out.

Isn’t musical improvisation live coding? Something that people like J. S. Bach were doing hundreds of years ago.

I don’t think that the TOPLAP (Transdimensional Organisation for the Permanence of Live Algorithm Programming) website is aimed at people like me, but I did enjoy the absurdity of its manifesto.

Who originated the term live coding? Perhaps it was a salesman trying to turn the poor user interface on the product he was peddling into desirable feature, i.e., having to write code to get the instrument to play a tune is a benefit.

Live performers are always looking for a new angle to attract the public. Perhaps they have noticed the various efforts to get kids coding and have adapted the message to their own ends. No, that cannot be true, academics are making arduous journeys to German country houses to discuss this emerging new technology Don’t worry if you missed out, you can attend the First Live Coding International Conference in July!

I’m sure live coding will get a few students programming who would otherwise have gone off and done something else. But then so would a few rainy weekends.

I suspect that most developers would prefer to do fewer live coding sessions.


My 2cents on the design of the contract language in Ethereum

January 12th, 2015 No comments

My previous post on Ethereum contracts got me thinking about how Ethereum should be going about creating a language and virtual machine for the contracts aspect of their cryptocurrency.

I would base the contract development gui on Scratch. Contract development will involve business people and having been extensively field tested on children Scratch stands some chance of being usable for business types.

For the language itself I would find a language already being used for contract related programming and simply adopt it.

At the moment the internal specification of contracts is visible for all to see. I expect a lot of people will be unwilling to share contract information with anybody outside of those they are dealing with. The Ethereum Virtual Machine needs to include opcodes that perform Homomorphic encryption, i.e., operations on encrypted values producing a result that is the encrypted version of the result from performing the same operation on the unencrypted values. Homomorphic encryption operations would allow writers of contracts that keep sensitive numeric values secret, they get to decide who gets a copy of thee dencryption keys needed to see the plain-text result.

The only way I see of overcoming the denial-of-service attack outlined in my previous post is to require that the miner who receives payment for executing a contract prove that they did the work of executing the contract by including program values existing at various randomly chosen points of contract execution.

Ethereum contracts, is the design viable?

January 11th, 2015 4 comments

I went to a workshop run by Ethereum today, to learn about cryptocurrency/virtual currency from a developer’s point of view. Ethereum can be thought of as Bitcoin+contracts; they are also doing various things to address some of the design problems experienced by Bitcoin. This article is about the contract component of Ethereum, which is being promoted as its unique selling point.

The Ethereum people talk so much about contacts that I thought once currency (known as ether, a finney is a thousandth of an ether, with other names going down to 10^{-18}) had been mined it was not considered legal tender until it was associated with the execution of a contract. In fact people will be free to mine as many ethers as they like without having any involvement with the contract side of things.

Ethereum expect the computational cost of mining to be significantly greater than the cost of executing contracts.

A contract is a self-contained piece of code created by a client and executed by a miner (the client pays the miner for this service). The result of executing the contract is added to the data associated with a block of mined currency (only one contract per block).

How confident can the client, and interested third parties, be that the miner paid to execute the contract is telling the truth and didn’t make up the data included in the mined currency block?

The solution adopted by Ethereum is to require all miners involved in the execution of contracts to execute all contracts that have been associated with mined currency, so that the result posted by the miner who got paid to execute each contract can be validated (51% agreement is required).

For this design to work, the cost of executing contracts to verify the data produced by other miners has to be small relative to paid income received from executing contracts.

If somebody is in the ether mining business, executing contracts has the potential to provide additional income for a small increase in effort, i.e., the currency has been mined why not get paid to execute contracts, adding the resulting data to blocks that have been mined?

Ethereum has specified the relative cost of operations executed by the Ethereum Virtual Machine. The Client specifies a value used to multiply these costs to produce an actual cost per operation that are willing to pay.

Requiring the community of contract executors to execute all contracts creates a possible vulnerability that can be exploited.

To profit from executing contracts the following condition must hold:

N_c A_o E_c < A_o P_o

where: N_c is the ratio of unpaid to paid contracts, A_o is the average number of operations performed per contract, E_c is the average execution cost per operation and P_o is the average amount clients pay per operation.

This simplifies to:

N_c {E_c/P_o} < 1

What would be a reasonable estimate for N_c, the ratio of unpaid to paid contracts? Would 1,000 miners offering contract execution services be a reasonable number? If we assume that contracts are evenly distributed among everybody involved in contract execution, then there are disincentives of scale; every new market entrant reduces income and increases costs for existing members.

What if businessman Bob wants to corner the contract market and decides to drive up the cost of executing contracts for all miners except himself? To do this he enlists the help of accomplice Alice as a client offering contracts at extremely poorly paid rates, which Bob is happy to accept and be paid for appearing to execute them (Alice has told him the result of executing the contracts); these contracts are expensive to execute, but Bob and Alice know number theory and don’t need a computer to figure out the results.

So Bob, Alice and all their university chums flood the Ethereum currency world with expensive to compute contracts. What conditions need to hold for them to continue profiting from executing contracts?

Lets assume that expensive contracts dominate, then the profitability condition becomes:

M_n N_c M_a A_o M_e E_c < A_o P_o

where: M_e is the increase in the number of contracts, M_a the increase in the average number of operations performed per contract and M_e the increase in average execution cost per operation.

This simplifies to:

M_n M_a M_e N_c {E_c/P_o} < 1

What values do we assign to the three multipliers: M_n M_a M_e? Lets say M_n=5,  M_a=10, M_e=3, which tells us that the price paid has to increase by a factor of 150 for those involved in the market to maintain the same profit level.

Given that Ethereum are making such a big fuss about contracts I had expected the language being used to express contracts to be tailored to that task. No such luck. Serpent is superficially Python-like, with the latest release moving in a Perl-ish direction, not in themselves a problem. The problem is that the languages is not business oriented, let alone contract oriented, and is really just a collection of features bolted together (the self absorbed use case for the new float type: “… elliptic curve signature pubkey recovery code…”, says it all). Another Ethereum language Mutan is claimed to be C-like; well, it does use curly brackets rather than indentation to denote scope.

If Ethereum does fly, then there is an opportunity for somebody to add-on a domain specific language for contracts, one that has the kind of built in checks that anybody involved with contracting will want to use t prevent expensive mistakes being made.