I spent some time talking to Paul and others about these topics at last year's E:Tech, but always as the between- or after-session banter. It will be great to have a whole conference dedicated to asking questions like (Paul's words) "where will alpha come from in the future? Will it be weather data? Data from Edgar filings? From the web? Or from social networks? Where are the entrepreneurial opportunities in becoming next-generation providers of financial data? Where are the real opportunities at the confluence of money and technology?"

Sounds great.

Its second accounting error in as many quarters will result in a $3.4 million restatement at Symmetricom Inc. that will push its third quarter into the red....The company said that error was related to accounting for outsourced labor related to installation work and resulted from reliance on a manually compiled spreadsheet that was not accurately completed. [emphasis mine]

When we talk about spreadsheet-based processes being prone to expensive errors, we aren't necessarily talking about things like flaws in design or testing, that will affect virtually any system and are generally a function of budget and schedule, though those are of course substantial risks. What we think is potentially more dangerous is the fact that complex spreadsheet-based reporting processes almost always end up being boiled down to a lengthy series of steps an individual has to do on a weekly/monthly/quarterly basis. People are generally very good at establishing these processes and ensuring that they are correct, but they are not generally well-suited to repeating them flawlessly. It's just not the type of work most people are expert at, and given the nature of spreadsheets, it is generally very difficult to tell that anything has gone wrong.

We feel the best way to deal with these issues is to automate where possible, and then scrupulously check results, which is much easier with automation. Set up a system correctly once, then ensure that the system performs the same process to your data every time. If the system can produce the required output quickly and gives access to intermediate calculations, validity tests and sanity checks can be easily run.

On the negative side, it's pretty clear that "mashups: the software" have aged better than "mashups: the music". During the cocktail hour on the first day, they had some DJs playing a set made up exclusively of mashups, as far as I could tell (I actually left. it was too much, but the DJs had something with the word "MASH" involved on their shirts, and it wasn't a stencil font separated with asterisks). Sure, I remember going to a party years and years ago and being like, "what the hell? Is that Christina Aguilera singing over the Strokes? That's hilarious!" It was hilarious. Once. Now its about as cool as fake Paul Thurrot.

I may have some of the details wrong here, but it will be clear soon enough when IBM launches this product suite for general consumption. It's great to see large companies like IBM professing a belief that better tools can dramatically change the way users build different types of software.

Theory P adherents believe that software can only partially be designed in advance. They believe that requirements suffer from observation, that the act of building software causes the requirements to change.

I always read stories like this with interest, and I’m well aware of the risks associated with spreadsheets. We discuss them all the time internally and with clients and some of those risks inspired key design decisions in Proto. That said, I think a lot of these analyst research reports, in their haste to eliminate spreadsheet risks, play a little too fast and loose with eliminating spreadsheet benefits.

I’ve written about this topic before on this blog, because its very near and dear to Proto’s heart. I still feel its best summed up by Ray Ozzie’s phrase, “edge-versus-center” tension.

In many of the organizations where industry analysts are talking about “[pushing] the task of managing spreadsheets into the hands of the IT department,” it is important to remember that many of the tasks the spreadsheets were built to handle were already nominally in the hands of the IT department. The spreadsheet gets trotted out when the details of the problem change, when the solution delivered by IT is slightly inappropriate to the actual business problem, or when a new application or analysis is needed on a schedule that's too fast for the already overworked IT department; when it needs to get done now.

In any organization, there is going to be a tension between the rigor and predictability of central administration, and the agility and effectiveness of edge-level control. Our belief is that adding rigorous centralized controls to spreadsheets will certainly succeed in making spreadsheets less dangerous, but it will also make them useless. Making spreadsheets safer by imposing a heavy centralized change-management process is like reducing a knife’s propensity to cut its user by completely dulling its edge.

We think that change management, auditing, permissioning, versioning, etc., have a great role to play in reducing operational risks associated with spreadsheets, particularly in areas with regulatory issues to consider such as pharmaceuticals research. However, we think there are larger strides to be made in giving people at the edge tools and techniques that enable them to build quick, one-off apps, while encouraging safer practices through design. It is important that the ability to rapidly solve new problems remain in the hands of business users, “shadow” IT people, and other people who are close to business problems. Overly centralization of planning and administration has a tendency to have disastrous results.

• The Pmarca Guide to Startups, part 1: Why not to do a startup
• The Pmarca Guide to Startups, part 2: When the VCs say "no"
• The Pmarca Guide to Startups, part 3: "But I don't know any VCs!"
• The Pmarca Guide to Startups, part 4: The only thing that matters
• The Pmarca Guide to Startups, part 5: The Moby Dick theory of big companies
• The Pmarca Guide to Startups, part 6: How much funding is too little? Too much?

We felt that there was a considerable degree of confusion around these topics. What are the salient features of a mashup? Who fits where in the technology stack and evolving ecosystem? Will apps characterized by the mapping mashups we've seen online ever have business value? We were frequently asked to make apples-and-oranges technical comparisons about products and companies in the mashup space. Without a comprehensive framework, these comparisons have at best been "round-ups" that broadly differentiate between products using a couple parameters.

With that in mind, we wrote a whitepaper that gives a brief overview of the space and also expresses our view on the future value of mashups in business and enterprise environments. Throughout the paper we build a framework that we hope illustrates what constitutes a mashup and how to evaluate different mashup products. At the end, we summarize this framework in 17 "evaluation criteria" and a corresponding illustration that anyone looking at a mashup technology for a business or enterprise application should consider when choosing which product(s) to use. It's our aim to clarify the roles of technology involved in building business mashups without making value judgments about specific vendors in the space. Please read it and let us know what you think.

A while back we did a blog post showing how Proto could consume a rich RSS feed made with Yahoo! Pipes to create an apartment searching and management application. I then learned about RSSBus from a Proto user’s blog, and I got excited about the opportunities that would open up if Proto could access RSS feeds of business data. There’s a good post by Prashanth Rai talking about this very topic and the enterprise information ecosystem of the future if you want additional background.

At ETech we had a chance to meet the RSSBus team and saw first hand how they deliver on turning proprietary data and applications into RSS feeds. Since Proto’s goal is to turn structured data and services into useful applications, I had to check it out first hand.

Here’s what I came up with:

In many businesses, client entertainment is a substantial cost. Managers, CFOs, and COOs would like to know if the $2,000 wine bill in Atlanta actually helped their salesperson close a deal. And since RSSBus can output feeds of the relevant data, Proto can provide the front-end to manage and report on the data sources to show the relationship between expenses and revenues.

To do this, we used RSSBus feeds that could be generated for:

1. American Express charges for a given user account
2. Salesforce.com Opportunities
3. QuickBooks Expense categories or a Google spreadsheet with a standard list of categories
4. A set of files with the Amex charges tagged by users (created using Proto) that link each expense to an Opportunity in a particular category
5. A feed of booked revenues for opportunities (either from Salesforce.com or QuickBooks)

The illustration below shows how these data sets come together:

Next, I'll walk through the specific applications.

Tagging the Amex Expenses with an Opportunity and an Expense Type:

This tool pulls recent Amex charges and puts them in a table for the end user to tag them with a corresponding expense category and opportunity. Since these end users aren’t expected to love this process, we made it as easy as possible by populating the list-box choices in the table with Salesforce.com opportunities and expense categories.In addition to making the process simple enough that it should only take 2 minutes a week this has the added benefit of keeping the data clean at the point of entry.

The same idea and infrastructure could easily be applied to an environment that didn’t rely on Salesforce.com opportunities and instead was a list of accounts, banks, artists, or any other client that is the beneficiary of entertainment. Anyway, you get the idea: the Amex expenses are now correctly bucketed and associated with revenue opportunities.

When the user is done, they push the results to a file (or database, or spreadsheet, etc.) so that RSSBus can provide a consolidated feed of correctly associated Amex expenses.

This feed along with the actual booked revenues, from Salesforce.com or QuickBooks to name a couple possible sources, are mashed together into two more Proto applications shown below.

1. Drill down report of total expenses and revenues.

This view shows the total revenue and expenses for each salesperson on the left hand side. Clicking on any bar in the graph further breaks down that salesperson’s expenses and revenue on a deal by deal basis. Clicking on one of those bars (on the right hand side) shows the individual American Express charges associated with the revenue. Again, the extensions are endless – but the idea is the same: by combining proprietary feeds of data, spending can be related to success.

2. Show expenses by date, colored by either Salesperson or Expense Category.

The next screen I threw together shows a distribution of expenses over time. Coloring the points by category shows trends in expense types or salesperson spending. Also, specific points can be drilled into to see the actual American Express expense details for large or otherwise unusual bills. Again, the possibility for extensions and modifications abound – these are just starting points to get ideas flowing.

In summary, we take data that will never be integrated into an enterprise-scale data warehouse, and we tactically combine it with the help of RSSBus into useful business intelligence and decision support tools. Is this type of ad-hoc integration the future of mashups in enterprises and SMBs? We think so.

CFOs and COOs get the clarity they need to optimize their revenue-driving entertainment and sales expenses. Salespeople don’t have to endure a painful or tedious process to categorize their expenses because the relevant choices and sources are pulled together for them in one custom user interface. Everyone wins.

• Good old Techcrunch


• Tom Gleeson's Gobán Saor


• Dennis Howlett's AccMan


• Timo Elliott wrote a piece about Proto and 'BI 2.0', something I've been thinking about a lot and meaning to comment in greater depth on.