Toomre Capital Markets LLC

Real-Time Capital Markets -- Analytics, Visualization, Event Processing, and Intelligence

Aldon Hynes's blog

Web Enabling MATLAB Analytics

Over the past several months, I’ve been working on several projects at Toomre Capital Markets LLC dealing with optimal ways of distributing results of calculations done in MATLAB. In previous posts, I’ve written about accessing and storing information from Excel and SQL databases. In this post, I want to look at distributing some of the results via web pages.

One approach that we’ve used is having a MATLAB routine run as a DLL called from an IIS web server. The webserver passes key parameters to the MATLAB routine. The routine then retrieves information from an SQL database, runs a set of analytical routines and stores the results back in SQL. The IIS server retrieves the results from the database and builds the webpage for the end users.

There is a lot that can be done using this methodology, but it can be complicated making sure that all the variables are properly mapped between Matlab, SQL, and the webpages. If you have an IIS server and an SQL Server up and running and have plenty of time, you can build a very powerful application this way.

Performing an SQL Style “Group By” in Matlab

On a recent project, I found that I needed to group data in one matrix by values in another matrix, but I could not find a nice way of doing this. So, I wrote a simple Matlab function to do this sort of grouping. I am describing it here in case others run into a similar need.

Let me start off by describing some sample data. Assume we have been collecting data on the number of cars that pass a checkpoint by hour for several days. We might end up with data that looks something like this:

Day hour Cars
1 8 47
1 9 38
1 10 35
2 8 52
2 9 42
2 10 37
3 8 43
3 9 34
3 10 45

In SQL, we might do something like:
SELECT day,sum(cars) from data group by day or
SELECT hour,sum(cars) from data group by hour

What I did was create a function, I call ‘bucket’ for gathering the data into buckets.

Is Sergey Aleynikov Really A Russian Spy Who Stole Trade Secrets That Could Cost Goldman Sachs Millions?

A top story of the day on many of the news outlets is about Sergey Aleynikov, the thirty-nine year-old former vice president who allegedly stole trade secrets from Goldman Sachs and stored them on a foreign server. The breathless headlines are staggering. Code theft could cost Goldman millions, US says, To Catch a Rogue Quant, Russian Said to Be Ex-Goldman Worker Charged in Theft and The Dumbest Man at Goldman Sachs.

As President Obama visits Russia, the homeland security, terrorism and anti-immigrant blogs are abuzz about the alleged Russian spy. You have to look hard to find the headline, Goldman sees no impact from computer programmer-source. It isn’t as exciting.

Before Aleynikov is hung for international espionage, I thought it would be good to dig a little bit deeper into what happened. According to the an affidavit by Michael G. McSwain entered into the Southern District of New York, FBI agent McSwain charges Mr. Aleynikov with “unlawfully, willfully, and knowingly, without authorization, copied, duplicated, sketched, drew, photographed, downloaded, uploaded, altered, destroyed, photocopied, replicated, transmitted, delivered, sent, mailed, communicated and conveyed, a trade secret that is related to and included in a product that is produced for and place in interstate and foreign commerce with the intent to convert that trade secret to the economic benefit of someone other than the owner thereof, and intending and knowling that the offense would injure the owner of that trade secret, to wit, Aleynikov, while in New York, New York, and elsewhere, copied, without authorization, proprietary computer code belonging to a financial institution in the United States and then uploaded the code to a computer server in Germany.”

Writing BLOBs from Matlab to SQL Server using ActiveX and Stored Procedures

Over the past few months, I’ve been working on a project that integrates sophisticated Matlab financial models with a Microsoft environment. Back in February, I wrote about writing from Matlab to Excel using ActiveX. Since then, I’ve been focusing on SQL Server and want to share those experiences.

The simplest way to access a Microsoft SQL Server from Matlab is to use the Database Toolbox. This toolbox allows for an ODBC or a JDBC connection to a database. Since our functions are running in a Microsoft environment, we went with an ODBC connection. The connection is very simple to open:

conn = database('ODBCdatasourcename','ODBCusername','ODBCpassword')

In our case, we would then ‘ping’ the connection to make sure everything has connected properly, and if not, return an error:

try
ping(conn) ;
catch ME
% insert error processing code here …
return;
end

We then execute our SQL and check the results

sql = ‘select a,b from c where d = 10’;
curs = exec(conn, sql) ;
curs = fetch( curs ) ;
NumRows = rows(curs);
if (NumRows < 1 )
% insert error processing code here about no rows returned
return ;
end

result = curs.Data;
for i=1:1:NumRows
if (~isnan(result{i,1}))
a{i} = result{i,1};
b{i} = result{i,2};
end
end
close(curs) ;
close(conn);

It has worked nicely for just about all of our needs. However, for speed and flexibility, we found it useful to use ActiveX for some of our database access.

#followfriday

@ltoomre @mrkwpalmer @ltabb @johnjcoulter @kw2dio The #SIFMA edition:

Since March, I’ve been posting #FollowFriday posts on my blog, highlighting people that I find interesting. Often, these posts focus on specific groups of people; friends from Connecticut, Connecticut news outlets, political activists, people doing interesting stuff in online marketing, and so on. This week, I’m highlighting a few people doing saying interesting things about financial markets.

As is my standard, I write it as a blog post, so people can get additional information on why I am following these people, and TwitterFeed sends the key part to Twitter.

Writing from MATLAB to Excel Using ActiveX

On a recent project we have needed to compare two versions of a complicated financial model. One is written in Excel and the other in Matlab. The model uses over 150 different input variables, so it can be a challenge to make sure that essentially the same variables are passed to both models. Today, I modified one of the routines to use ActiveX to send data from Matlab to Excel. The process was not easy to figure out, but once I figured it out, it is extremely straightforward.

Initially, we were using the Matlab command xlswrite. It is a quick and easy way to write data from Matlab into Excel. It uses a COM server, and can be a little slow, especially if you are making multiple calls. However, if you want to do anything more than simple reading and writing of data to a spreadsheet from Matlab, you need to start using the ActiveX server functionality that Matlab supplies.

Ad:tech Tales from the Bleeding Edge

Before heading down to ad:tech yesterday, I paused to wonder if it was worth it. All the conferences tend to look and sound the same. Rarely does a panel ever live up to the ‘tell me something I don’t know’ request that so many writers seek. Tales from the Bleeding Edge was different. It was the best panel I’ve been to in ages and Lori H. Schwartz, SVP and Director of Emerging Media at IPG Media Lab deserves kudos for putting together such a panel. The write-up alluded to all kinds of cool technologies, many of which were not touched upon, but the ones that were, were wonderful.

Bill Capodanno, Director of Digital Marcom Planning and Effectiveness, Microsoft led off with a brief discussion of Microsoft Surface. He asked how many people in the audience had heard of it and a majority had. He spoke about a brief video they did about Microsoft Surface as well as the parody that was produced. It is a very funny parody, well worth the watch.

Is Directors’ and Officers’ Liability Insurance Coming to Second Life?

In recent days, there has been a lot of activity as investors in Second Life corporations attempt to recover assets from failed companies and their owners. The actions, as well as discussions surrounding these actions raise the question of whether Directors’ and Officers’ Liability Insurance might be coming to Second Life.

Many people view Second Life as a three dimensional gaming environment. Others, recognizing the value of the immersive environment look to set up advertising in Second Life or to use it as a fancy new way to hold online conferences. Yet many overlook the importance of the Second Life currency and the evolution of financial markets in Second Life.

Second Life and Streambase Convergence

It can be a wonderful things when things come together just right. As an example, I finally got around to upgrading my antiquated cellphone, and I can use my new phone as an MP3 player and a video camera. It makes my life easier and much more fun.

Today, I heard about two other interests of mine converging. This is a much geekier convergence and I’m trying to find out if it too will make my life easier and much more fun. What I’m talking about is the press release announcing Linden Lab Selects StreamBase’s Event Processing Platform To Power Real-Time Analytics And Reporting Throughout the Second Life Grid.

Cool! Will I get to play with that?

Exploring the Potential of Drupal, Second Life and Complex Event Processing

The online experience is changing from a world where users pull static text off of webpages to a world were the experience is much more immersive with pictures, videos and three dimensional animations and much more interactive with Web 2.0 functionality, instant messaging and real time data. As these changes take place, information providers need to rethink their online strategies and how they use various tools.

At the center of any information strategy is a good content management system. Drupal is a widely popular open source content management system that facilitates the organization and presentation of information. It allows users of the site to easily add content and has been expanded to include better functionality for images and video. It is this ability to easily be expanded that makes it interesting as more immersive synchronous environments become more popular.

One such environment is Second Life. Second Life has been getting more and more attention as companies explore how they can use to achieve corporate goals. With Second Life, you use a Second Life client, instead of a web browser to access the Second Life servers. These servers provide a three-dimensional real time environment where uses interact with objects that have been created as well as with one another. Second Life has it’s own currency which facilitates micropayments there and there is an active currency exchange to change the Second Life currency, called Linden Dollars, into U.S. dollars.

Working with Jim

Back in January, I wrote a blog post about managing transitions and a new company called CenterNorth. They specialize on the human side of technology leadership and work with companies of all sizes.

Today, I found an article about Leadership Near the Summit. It speaks about an MIT grad climbing his way up to CTO and then facing a difficult annual review. I saw many aspects of myself in that article and at Toomre Capital Markets, we work with some of the brightest engineers out there. We see stories like this all the time.

I’d encourage you to read the article, and if you see characteristics of yourself or people that you’ve worked with in this article, please contact CenterNorth to assist them in their qualitative research into the “best practices for leaders of technical all-stars” You can respond in writing or they will talk to you one on one and will keep all data you provide completely confidential, even in the write up of the report.

Web 3.1: Real Time Enterprise Internet Presence

Between Twitter messages about SXSW, video streams and online text chats about Freedom to Connect, Blogging the Libby Trial, and preparing for theNew England News Forum, I’ve been spending a lot of time thinking about the Internet and I want to share some of these thoughts in terms of the financial services industry as well as a broader context.

Building a Second Life Ticker Plant

There is a lot of data available out on the web that could be scrubbed and analyzed to give firms competitive advantages. Lars has worked with weather contracts and I spoke with him about methods of gathering and scrubbing that data. I’ve spoken about Second Life as an emerging market place with data that could be scrubbed and analyzed as well. It provides a good starting point to explore how to scrub and analyze online data.

In my post about foreign exchange trading in Second life, I asked about getting real time data from Second Life. I mentioned http://secondlife.com/xmlhttp/secondlife.php as a source for some information in XML format that could be used. Yesterday, I wrote about 70 lines of PHP code which retrieves, parses and stores the Second Life statistics into an SQL database. Below is a slightly geeky description of what I did, which illustrates some of the issues your run into.

Pursuing the elusive micropayment

In 1989, Brewster Kahle introduced his Wide Area Information Server Concepts. In many ways the web is now being used in ways that Brewster was suggesting for WAIS. Included in his suggestions were issues of how people would get paid for the content they put online.

Today, people are still working on different ways to get paid online. You can download music online complete with digital rights management for a buck from various sites, if you sign up ahead of time. You can subscribe to various newspapers online. Yet the problem of micropayments still hasn’t really be solved.

The future of the New York Times

The creators of Epic imagine a world in the not too distant future where the New York Times has gone offline. Yet in an interview with Eytan Avriel (NewsTrust review), Arthur Sulzberger of the New York Times, Sulzberger talks of a day, perhaps even sooner when the New York Times publishes only online.

The article talks about losses in the newspaper industry, including “a $570 million loss because of write offs and losses at the Boston Globe. “ Yet provides the interesting observation that costs of moving to an all digital platform “aren't anywhere near what print costs” are. As cost and revenue structures change, information distribution organizations need to rethink what they are doing.

Interestingly, the vision that Sulzberger describes of the New York Times as “curators of news” is perhaps not all that different from a rosier view of what Epic presents. At Toomre Capital Markets we do our own curating of news related to financial services innovations. Yet the thing that seems to be missing from Sulzberger’s view is the role of community.

Yes, there is a community of readers, but how much do they interact? There is a community of sources that reporters use, but too often that community seems particularly small and disconnected. Interesting news and ideas emerge when different parts of community get linked together. It is part of what we try to do at TCM. It is part of the wider sets of social networking sites, and if the New York Times is going succeed, it is something they need to focus on as well.