We have just uploaded a new introductory video to gTrade.
You can also watch it on YouTube: http://au.youtube.com/watch?v=cCxkVhBVoC0
And Download the Original in M4V Format: http://www.gtradenet.com/videos/gtrade_intro_video_v1.m4v
– Guy
CEO & Co-Founder – gTrade
When you throw it and it comes back, its called a boomerang. Boomerang also happens to be the internal codename for the securities exchange engine powering gTrade. But lets wind back a bit. Securities exchanges are traditionally distributed systems. While we typically associated “distributed” with clustering, fail-over, scalability and so on, “distributed” in this case is not to be interpreted in a good way. All it means is that any single component within the traditional exchange infrastructure is too brain-dead to facilitate an exchange on its own. This is not so say that there are no provision for redundancy, en contraire, but this is achieved by clustering many instances of the same… brain-dead… component. So what does it mean, to be “brain-dead” then?
A distributed system is one made of a number of physically disparate components working together to achieve a common goal. That’s where the definition stops, and where confusion begins. You see, there are two ways you could achieve what I’ve just stated. Suppose you were building a distributed system that needed to perform tasks A, B and C. Alice could build three components, each performing the roles A, B and C respectively and networking the three. To achieve redundancy, she could run two instances of each of the A, B and C components, and networking the six. Wally, on other hand, could built a single, albeit more sophisticated component that could simultaneously assume the roles of A, B and C, i.e. it could handle all three tasks. For redundancy, he could run two networked instances of this component.
At the end of the day (month?… year?) both Alice and Wally have themselves a distributed system. Which is better?
Let’s consider an order that needs to be processed by each of the functions A, B and C in some sequence. Alice’s system would delegate the order to one system, then migrate it on to the second and, finally, to the third. Because the systems are networked, the network delay cannot be taken out of consideration, and the turn-around time would be substantial, regardless of the throughput capacity and present loads on the servers. Wally’s system could facilitate the order within the confines of a single server. Sure, the load on the server would inevitably increase, hampering the capacity to process subsequent orders, but with proper load balancing we can simply spread the orders across the remaining servers, adding servers as we need to.
Back to the original question. Which is better? At this point an overwhelming majority of literature on architecture and computer science tries, in order to be seen as politically correct, to take a neutral stance on the two solutions. “There is no correct answer…”, they would say. “It all depends on…”, they would add. Hell NO! Not here! Not on this blog! Wally’s solution is better hands-down, regardless of which way you look at it. Why? Because his avoids unnecessary network hops that plague today’s distributed systems, spreading functionality across nodes purely for the sake of looking and sounding “distributed”. The result, you guessed it: “brain-dead” nodes that are incapable of standing on their own two feet and accomplishing anything in the absence of their peers. Wally’s is distributed in every regard, yet it can run with as few as a single component.
How does this relate to a securities exchange (SE)? SEs for various historical, political and commercial reasons have erred on the bad side of the distributed paradigm. There is a separate match-up engine, a clearing house and a stock registry. Not to mention the various brokerage systems that cloud around the SE because everyday investors simply aren’t admitted directly into an SE infrastructure.
The Boomerang engine backing the gTrade securities exchange does everything you’d expect of a stock exchange: a proverbial “shampoo and conditioner, all in one bottle”. The clustering aspect, along with the benefits of scalability and redundancy is outsourced completely to DTS/S1 ‘Pitch Black’: a native Java transaction-oriented object database from Obsidian Dynamics. While it is less than a year old, and not yet available to the public, DTS/S1 has already proved its worth. DTS/S1 allows applications to manipulate plain old Java objects in a way that is isolated, durable, atomic and consistent, as you would expect of a transaction processor. DTS/S1 supports the ’serialisable’ level of transaction isolation, which is the highest level of transaction isolation achievable. Simply put, you read and write to your Java objects as you would, and those changes are persisted to permanent storage without any further intervention on your behalf. More so, the changes are documented in an audit log and can be traced at any later point in time. What were the changes? Who made them and when? DTS/S1 can provide the answers when asked. Most beneficial to Boomerang is the fact that DTS/S1 is clustered internally. It runs on as many (or as few) compute nodes as required, and can write data to a clustered NAS. And its fully transparent: Boomerang still thinks that it runs on a single machine.
The gTrade business model, and hence the Boomerang workflow, is based upon a two-phase company lifecycle. Let’s come back to this later. A traditional stock exchange facilitates the trade of equity in a publicly listed company. That is, it assumes the company has made its public offering and the brokers have distributed the shares among the interested investors. Some investors eventually express a desire to sell, while others are poised to buy. This is all well and good for a well-established company, but what does this do for a start-up? The short answer is nothing. Start-ups are on their own when it comes to gathering the initial capital and working hard for a number of years to get to a point where they are fit for public listing. This also happens to be the exit point for private investors, such as Venture Capitalists who have chosen to invest in the company from the outset.
gTrade allows for everyday investors to “subscribe” to a start-up by purchasing a number of shares at a fixed price. A start-up company that passes the necessary due diligence and is considered by gTrade to be fit for listing will “expose” a part of the company for public investment in exchange for cash which is used to bootstrap the venture. For example, company StarSoft might be willing to forfeit a 40% share in return for $100,000 in cash. They could break up the public share into 100,000 tradable class A shares, fixed at $1 each. StarSoft would be listed on gTrade and given, say, 3 months to raise the capital. Investors choosing to opt-in would “subscribe” to the shares at a fixed price per share. Remember, the clearing house and the stock registries are integrated into Boomerang, allowing it to hold both the shares of StarSoft and the investor’s money in escrow, until the company is fully subscribed. In that event, Boomerang will transfer the shares to their new owners, and the cash to the proprietors of StarSoft. On the other hand, if StarSoft fails to gather sufficient investor interest by the nominated due date, and the latter is not extended, Boomerang will recover the subscriptions by yielding the cash funds back to the investors and forfeiting the shares back to the start-up, in effect reverting all that has happened within that seeding round. The favourable outcome for everyone is, of course, that the company manages to fully subscribe and the initial cash-for-stock trade is settled. At this point, and for this particular seeding round (for there could be several rounds), Boomerang switches its mode operation to a conventional, fully-automated stock exchange, not unlike NASDAQ.
Boomerang’s fully-automated matching engine and the integrated registry allows investors to trade shares in real-time, and to receive and/or forego stock and cash the moment right after the exchange has been facilitated by Boomerang. And because both the stockholder registry and the clearing house are intrinsic parts of the trading platform, Boomerang allows for optional cash and mandatory stock reservations to guarantee delivery after an exchange takes place. One of the key differentiators of an integrated systems is that activities such as naked short selling can be eliminated in their entirety. At any point in time, Boomerang is fully aware of each of the investors’ holdings, including both cash funds and their tradable stock and subscription portfolios. This allows the punter to make optional cash reservations for buy orders, ensuring his/her transaction succeeds given a matching sell order. At the same time, Boomerang applies mandatory stock reservations for all sell orders, ensuring that stock is in the physical possession of the seller, thereby guaranteeing delivery. This removes the possibility of an unscrupulous investor or stock broker from orchestrating a naked short sell.
At gTrade, there is no decoupling of the investors and their holdings. This includes their cash funds. In addition to maintaining an investor’s personal information, their investment history and their stock and subscription holdings, gTrade houses what is arguably the investor’s most treasured possession: their wallet. You cannot complete a subscription or a buy transaction without having some prior funds in your wallet. Similarly, when a sell transaction completes, money will be deposited into your wallet. On the outside, you interact with your wallet by depositing and withdrawing money; you can “top up” and “sip off” from your gTrade wallet at any time. A wallet can be seen as a pool of money that can be invested, and into which the capital gains from the investments flow. There are times, however, when not all of the money you have in your wallet is “available” for investing. This occurs if you have chosen to reserve funds in a previous transaction. For instance, suppose I have $1000 in my account, and I’m in desperate need of 100 shares of XYZ and I’m willing to pay at most $1 per share. I could put in an unreserved buy bid, and providing a willingful seller emerges I have at least $100 in my account at the time of the exchange, the purchase will go ahead. But suppose I’m engaged in multiple buys and sells at the same time: money is flowing in and out of my wallet and while I’m doing quite nicely, there is no guarantee that there will always be $100 in my account to facilitate this “special” trade, and when the right seller emerges, I simply might be out of cash.
gTrade’s Boomerang engine has facilities available to exercises more fine-grained control over your buy orders should you decide that certain orders are more important than others. In the previous example, I could have reserved $100 (or less) for this trade. This amount would still reside in my wallet, but be locked to all other operations but that particular buy order. Funds reservations are a very powerful feature of gTrade, however, they’re somewhat restrictive in that they lock down a portion of your funds. To make this feature more appealing, we have enhanced it with what we call Early Funds Release (EFR). EFR is a mechanism that reduces the amount of reserved funds to an absolute minimum, and continuously re-evaluates the reservation figures, optimising them for your current buy position. EFR is best explained with an example. Suppose we have reserved $100 for an at-limit purchase of 100 shares of XYZ at $1.00. Later, we’ve been fortunate enough to find a seller who’s willing to part with 50 shares at $0.80. We’ve acquired 50 shares, spending only $40 out of our reserved $100. After the trade, Boomerang will evaluate our current position. It will see that we’ve still got 50 shares to fill the order, which will cost at most $50, given our initial limit. Yet we still have $60 left of the reservation. Boomerang will then release $10 back into the available portion of our wallet to be used immediately with any other offer.
Finally, I would like to point out that while gTrade will at some point support external brokers via a public API, the integrated nature of the platform makes its features available to the public through our Web desktop, which we have apply named “Webtop”. Webtop is our AJAX-based rich client that takes stock trading to a whole new level; not yet complete, but well on its way, and we hope to show it off in a couple of months.
Emil Koutanov
Chief Technologist and co-founder
gTrade
In order to build a stock exchange and trade investors money, you really need to be sure of the transaction engine underlying the platform. gTrade utlises the next generation of transaction engines to ensure not only performance and reliability, but also meet the extreme regulations imposed by the Investment Protection bodies such as the SEC – this includes things such as the retention of data and its auditability. You simply can’t do this with your run of the mill MySQL database and Ruby on Rails application – no matter how well you implement the system. Don’t get me wrong, the LAMP (Linux + Apache + MySQL + PHP) & MARS (MySQL + Apache + Ruby + Solaris/Linux) stacks are great. They are better than that, they’re excellent – totally revolutionised web development. However, those types of stacks were never designed from the ground up for an enterprise implementation – this is evident from the great efforts needed to cluster MySQL and make RoR highly available – often ending in failure or extremely complex and rigid implementations.
This is where gTrade differs – we use Obsidian Dynamics’ DTS/S1 ‘Pitch Black’. DTS/S1 is a next generation distributed transaction server and native object database. Pitch Black is a clustered solution that embraces high levels of parallelism to achieve absolute reliability, fault tolerance and organic scalability resulting in ultra high performance transaction processing. Parallel to this, Pitch Black achieves the lowest cost per transaction of any other commercial transaction processor or transactional database management system.
Pitch Black is all about simplicity and speed. Where relational databases bear the overhead of ‘unpacking’ complex object-oriented structures to fit a 2D table model, and then ‘repacking’ these structures to present back to the application, a native object database persists business structures as-is. The net effect: where peer products require million-dollar computer hardware to achieve speeds in the order of tens of thousands of transactions per second (TPS), Pitch Black can achieve similar figures on hardware that would take up one unit of rack space. Given clusters of high-end server hardware, coupled with clustered network attached storage (NAS) devices, speeds in excess of 100,000 TPS are achievable.
While Pitch Black has been designed to extract blistering performance from a minimal deployment, it has been engineered for growth. Pitch Black assumes an N x M (N-by-M) compute and storage model, where N clustered DTS/S1 nodes operate on a shared storage facility that may internally be clustered to M nodes. Although not novel, a modular shared-disk approach is far more applicable to systems such as DTS/S1 which advocate the proximity of business logic to storage. Its elegance is in allowing an organisation to scale either its computational power, or storage capacity, or both, whichever proves to be the bottleneck. This level of scaling is a physical impossibility in a shared-nothing architecture, where each cohort is both a computational and a storage node.
 |
Pitch Black employs a microcontainer architecture, which provides additional flexibility in the interaction between business logic and the storage subsystem. The microcontainer provides a number of services to the application, which include (predominantly) a transaction service, resource locking and deadlock avoidance, querying (of both the overall state, historical state and the transitional data) as well as an external interface for exercising queries and invoking transactions remotely. A significant advantage of the DTS/S1 microcontainer is its ability to host and execute the full application business logic suite within DTS, and distribute the processing load throughout the nodes that comprise the Pitch Black cluster. Unlike stored procedures employed by most commercial databases, DTS does not lock the developers into working with proprietary languages that only the database understands. Because the DTS transaction and object broker service are native, the application developers are not required to learn any other scripting language. This paradigm extends to querying DTS – like any other aspect of the system, the queries are also written in the developers’ native language. Presently, DTS/S1 supports the Java(TM) programming language, including Java ME and Java EE technologies.
 |
The DTS/S1 transaction service is subdivided into two layers – the (high level) transaction broker and the (low level) storage and interconnect fabric. The higher level facilitates the principles of the DTS model, including object revision control, decoupled transactions, recovery protocols, remote transaction and query brokerage, an API to interface with the business logic, resource locking and others. Its unaware of the interconnect and storage protocols that are provided by the pluggable interconnect fabric (PIF). This enables Pitch Black to support a variety of physical storage platforms, including storage area networks (SANs), network attached storage (NAS) serving a POSIX-compliant file system, non-volatile RAM (NVRAM) and FLASH technologies – any storage implement ranging from enterprise-grade storage farms to embedded systems.
The next level up in the DTS stack is occupied by transactions and queries. These are always defined on the DTS node in form of conventional Java bytecode. They may be either a part of the business logic that resides and executes on the DTS node, or “named” parametric transactions and queries that can be invoked remotely via the Remote Object Model Broker (ROMB) API through the external interface.
Like many other things, the DTS microcontainer does not impose structure on the external interface, which may be a pathway for communication with other components that form part of the system, or a B2B portal for external 3rd party systems.
We have christened the Pitch Black design model as a “floating objects” architecture. All objects in DTS undergo separate life cycles, however, operations may atomically combine fragments of certain objects’ life cycles in time. Although independently versioned, objects may reference others forming arbitrarily complex graphs. Because Pitch Black is clustered and each DTS node is inherently multithreaded, there are avenues allowing data to be viewed and manipulated in parallel. Recently accessed and modified objects are cached on each node, and cache coherency is maintained by each node.
The floating objects architecture assumes the transparent migration of objects from one node to another, where objects may simultaneously co-exist on multiple nodes for read-only operations. In order to accommodate this, DTS provides a re-entrant read/write lock service and deadlock avoidance mechanisms which are always employed by DTS/S1 but are also available to the application level. By comparison, relational databases are unaware of the individual objects that form the data tables. The only way serialisable transactions (the highest level of transaction isolation) are achieved is through row-level locking, which in all but the most trivial cases has to acquire a series of locks, as multiple rows across several tables may constitute a single object. This is largely uncontended synchronisation, where locks are acquired out of need. Being aware of the higher-level object context, Pitch Black contends for the minimal number of locks to achieve the same level of transaction isolation.
The next design fundamental behind Pitch Black is decoupled transactions. Pitch Black records all data manipulations as transaction objects. Each transaction object depicts the change that one or a group of objects must undergo atomically. Should a transaction fail to update all objects in its working set, it will roll back all changes, returning the system back to the precise state it was prior to the commencement of the transaction. Pitch Black maintains a coherent cache that ensures that all nodes within the cluster are exposed to a consistent overall data state. The fundamental distinction between Pitch Black and peer transaction processors and databases, is that the latter tend to have monolithic data structures that encompass the overall system state and get persisted periodically.
In order to illustrate the difference between monolithic and floating object models, consider the following diagram that illustrates the evolution of an object in a relational data model. A relational data model depicts all data as a set of tables. Most enterprise business objects map to more than one table and, conversely, a single table depicts fragments of more than object. Relational databases operate at row level, and being unaware of the object schema cannot exercise revision control on a per-object basis. What we are able to observe then, is the evolution of monolithic structures in time, where each monolith is assigned a revision number following a transaction.
 |
All transactions ultimately relate to the same monolithic data structure and must be replayed from the journal in strict sequence to rebuild a transaction-consistent view of the overall state in an event of a failure. During recovery, the system will first inspect its stable storage to locate the most recent image taken prior to failure. The image will then be loaded into memory and the system will re-evaluate each transaction recorded after that image up to the point of failure. Transactions that have not been marked as committed will be rolled back. Theoretically, after this process is complete the system will assume the same state as it was in just prior to failure.
Re-aligning a single object invariably involves a complete replay of all pending transactions – a process that takes minutes if the system was heavily loaded prior to failure. Pitch Black operates on a finite set of lightweight objects that can undergo independent versioning and may form completely isolated chains of transactions that may be replayed in parallel and out of relative sequence. This is illustrated in the figure below:
 |
At first glance, we may be mislead in assuming that this diagram depicts a set of wholly interrelated objects. In fact, we can trace out two completely decoupled transition paths with no overlapping intermediate dependencies (red and green on the diagram). It follows that to recover object D from the initial set of objects, we only need to replay two out of the 5 transactions depicted in this graph.
Pitch Black employs a complex algorithm that builds a dependency graph and isolates non-overlapping transaction chains that are replayed on demand – depending on which objects need to be re-aligned with their true state. Compared to the traditional roll-forward replay model this is blazingly fast!
The concept of decoupled transactions incidentally lends itself to the same observable behaviour as that of JIT compilation. Every other transactional storage facility employs an ahead-of-time recovery protocol, where the first post-failure action is to completely re-align the overall state with the journal. A system concurrently logging at a rate of 10,000 TPS, for example, falls behind the transaction-consistent state by precisely 10,000 transactions for every second that it delays synchronising the data images with the journal. When a failure occurs, the entire log is replayed. Because transactions are strictly ordered, the replay is performed sequentially, which forbids the system from operating until its state is completely re-aligned. Consider Pitch Black: although it can in principle, DTS does not materialise every single transaction from the journal into the object images. It is, in fact, flexible in how it synchronises the journal with the object images. The maximum allowable journal backlog can be tuned for higher throughput, which would ordinarily incur a significant recovery time penalty on every other system. Not for Pitch Black, where a decoupled transaction model enables it to operate on small fragments of the overall journal backlog to instantaneously recover only those objects that are needed at the time of the recovery. Remaining objects that are not required immediately following a failure can be recovered later, either on demand, or at the discretion of Pitch Black.
Apart from achieving unmatched throughput and near-zero recovery times, decoupled transactions have overturned the classic parallel computing problem: cache coherency. A high-throughput, single-node database has one problem to solve, that is, when to synchronise between the journal and the images. A clustered database is challenged by a second problem, when and how to update the object caches on its peer nodes (cohorts). Typically, this would involve some use of the communication fabric that provides the interconnects between the cohorts. Given a sufficiently high throughput, this places an immense load not only on the interconnect fabric, but also on the processing capacity of the cohorts. The JIT-recovery nature of decoupled transactions provides any Pitch Black node with an opportunity to generate a transaction-consistent image of an object without the participation of its previous owner. Faster-than-light transaction processing is here, and its black – Pitch Black.
Pitch Black has been designed to unify all enterprise data storage requirements into a single, cohesive solution. We have achieved this by closely mimicking the heap allocation model of memory-managed development environments such Java(TM) and similar programming languages, and depicting similar object-lifecycle semantics in a persistent storage model. In plain terms, what this amounts to is the ability for application developers define classes as they normally would, instantiate those into objects, manipulate various fields and have those changes transparently committed to stable storage. The objects themselves are plain Java objects, potentially existing objects that already depict the business data, that have been enhanced by implementing a lightweight interface defined by the DTS/S1 container.
Pitch Black persists all data on an object-by-object basis, whereby each persistent entity is subject to separate revision control and system-wide locking. Because an individual object is so small by comparison to the entire organisational data store, DTS/S1 affords to make near-instant, permanent modifications to objects soon after it has committed transitional data to its recovery journal. Should one or more nodes in a DTS cluster fail at any point, the remaining nodes will absorb the increased load and provide immediate fail-over. The most recent transaction-consistent data state will immediately become available to the remaining nodes and the objects can be utilised immediately without incurring the penalty of a roll-forward replay.
Modified objects undergo revisions, where all previous revisions are retained and can be queried directly without reverse-replaying transactions to gain a historical perspective on the data. This is a yet another distinguishing factor that cannot be attained by competing products since they only store the most recent data image and a change log. Pitch Black can be configured to store checkpoint object images for each and every alteration of the data. Since data is checkpointed on a per-object basis, it is trivial to isolate areas of the system that may require frequent historical trend inspection, such as an organisation’s monthly sales/performance figures. These snapshots can be consulted directly, without the need to backtrack the journal or set up a separate data repository.
Two areas of computing have served as the primary influence for the design of Pitch Black. The first is managed-heap programming environments and adjacent technologies such as type safety, pointer arithmetic and garbage collection. Even JIT (Just In Time) compilation was not overlooked. This is closely approximated in how DTS stores and manipulates its objects internally. This also reflects how the application interacts with DTS.
The second area is revision control systems (RCS). A modern RCS, such as the popular Subversion, are highly transactional in nature. While they offer per-resource revision control, they also support bulk operations and an all-or-nothing atomic commit. These operations are fundamental in transactional object databases. But apart from these primitives, RCS systems expose historical views, differences and change sets. This is not the norm for transactional systems, but it is in Pitch Black.
Check out DTS/S1 at: http://obsidiandynamics.com/dts/
– Guy
Saving the Venture Capital Market
With the National Venture Capital Association (NVCA) releasing their report on their predictions for venture capital in 2009, some key points resonate with the driving forces behind gTrade.
The first point is the most obvious one – capital in drying up…. fast!
“While 96 percent of VCs predict that more venture firms will not be able to raise money in 2009, a lower percentage, 85 percent of respondents, believe institutional investors will reduce commitments to venture capital asset class.”
96% is a significant portion of the VC market. This will result in a significant reduction in the number of investments being raised, and as a consequence the Web 2.0 engine of social sites will almost certainly grind to a halt. This is where gTrade is targeting to help by raising capital from investors who have lost trust in traditional markets and would like to share their wealth with these emerging businesses with a vision of a nice return if the company succeeds.
The NVCA article goes on to say:
“While many existing institutional investors are struggling with their allocations and future investment decisions, we will see new limited partners, many from overseas, enter the U.S. venture capital industry, said Heesen. “Despite the fodder, we do not anticipate massive failures of limited partners to make capital calls. Many will sell their positions on the secondary market out of necessity. Yet, that will just change
the mix and allow other institutional investors access to funds they could not access in prior cycles. High quality venture firms will be adequately funded going forward.”
Perfect – gTrade is designed to allow institutional investors and regular investors (you & me) connect with the businesses that need the startup funds. We are then able to use our collective knowledge to value and assess the likelihood of success for these startups, and support then financially and morally.
The larger and more critical issue as I see it, is the liquidity issue with current and future VC funding – which simply doesn’t exist. “An overwhelming number of venture capitalists (72 percent) do not expect the IPO market to re-open for portfolio companies until 2010 or beyond. A more optimistic 18 percent see the market opening in the fourth quarter of 2009. While venture-backed acquisition volume is expected by 57 percent of venture capitalists to remain the same or increase, 87 percent of respondents predict that acquisition transaction value will decline.”
With exits getting more and more difficult, so in turn does entry. The capital that would otherwise cycle around the VC market from matured startup to emerging startup is just not happening. We at gTrade are addressing this issue directly – there is no need to exit the market. By trading in a free market, there is no need or requirement to exit immediately – institutional investors and regular investors alike can exit and enter investment as they see fit.
So as you can see gTrade has been predicting for some time now what the NVCA is now predicting for 2009 and beyond – it feels good to finally have some support from a large industry body. We are hoping this will be the catalyst for existing, and maybe is some way monolithic, venture capital funds to come to the party – the gTrade party – for the new way to raise and trade venture capital. This may just be the saving grace for the current VC market.
It’s about time for another update on how we are progressing here at gTrade, and we are making some great advancements I really want to share with you. It has been a little while since our last update – but we do have an excuse being holiday season and all.
As some people are visiting this blog for the first time, I will do a quick recap on what gTrade is: we are a Web 2.0 company that promotes technology start-ups. We represent the next generation securities exchange: not just trading conventional equities, but enabling everyday investors to purchase shares in companies that have not yet listed publicly. As opposed to traditional venture capital, gTrade brings a start-up company to the collective attention of millions of Internet users, letting the people decide in a start-up’s future by investing their own money.
Alright, back to the platform – to wet your appetite a little, I have taken some screenshots of the trading platform. We have tried to keep the interface clean and simple, and responsive like a fat-client side application. This has been achieved through using Jquery and good old AJAX (well AJAJ – Asynchronous JavaScript and JSON).
The first page you see at login is your ‘Dashboard’ – your one-stop-shop to do everything and go anywhere on the trading platform. Our ‘Update’s box at the top of the dashboard is a personalised news feed for you. It will show updates for such things as update on the companies you have invested in, their twitter feeds (and other Web 2.0 news feed technologies), up coming Initial Public Offerings and gTrade announcements.
gTrade Dashboard Screenshot
We have also given you the ability to create ‘Watchlists’ to monitor listings on the platform that may be of interest to you. You can then subscribed to RSS feeds for each or all of these watchlists and monitor then from anywhere on the net.
Those with a keen eye may have noticed a little box called ‘SmartBox’ – stay tuned, I will go into more detail in a future posting.
On the right hand side of the page there is a ‘Context Area’ box – this changes depending upon what page you are on and which stock you are looking at. This gives you a quick look at fast information – such as market quotes, links, events and historic transactions.
gTrade Buy Transaction Screenshot
The above screen shot is an example of a buy transaction. Here you can see Apple Inc stock is being purchase – at an extremely low price 
When the company name or stock code is being typed, an autocomplete box slides down to assist with company selection. Once selected, the context area is loaded with information about the select listing. This gives you an instantaneous update on the company and assists you in making a more informed decision with the right information at the right time.
You have the option to buy at a set price or fill the order as quickly as possible by using the ‘buy at market price’ function. This function will buy the next available stock that becomes available provided you are at the head of the queue – great if you want to jump in quickly just before you think a company is going to go up in value. You can also set the validity duration for the order – how long untill it expires. This can be from 1 hour to 29 days.
To give you a hand, an estimated order total is calculated for you. This is a rough estimation and can deviate greatly if you choose market price transaction – this is because you can buy stock at any price.
So when is this becoming available to the public? Soon!! We are working very hard on making the platform as stable, reliable and have greater cross-browser compatibility. We are doing some internal testing at the moment and ironing out the browser incompatibilities. A ‘game’ version with pretend money and companies will be announced in the not too distant future to help with beta testing – you can subscribe from our homepage – http://www.gtradenet.com
Till next time,
Guy
The next database won’t require a schema. It won’t require special tools to manage the tablespace. It won’t require a JDBC connection and it won’t require an ORM framework. You’ll give it an object and it will store it. You’ll ask for an object and you’ll get one back. Most importantly, it will be intrinsically aware of transactions. You’ll be able to query transactions, audit them and report on them, construct statistical charts: without a single line of code.
Let’s go back a step. Why do we need a schema, and why do we need a O-R mapping? Because databases unwrap objects into flat tables and we need a way to repackage them on retrieval. Eliminate the packaging problem and you’ve solved about 50% of the impedance mismatch problems that exists today. This is nothing new – it has already been done with native object databases.
Why do we need a connection to the database? Because a database resides on a separate machine or, at least, executes in a separate process. Why? Because databases are too slow to run on the same machine as the application server in production. If they would, the whole system would just become bogged down with serving users and storing data. Solve this problem and you’ve eliminated another 25% of the issues. Keep the connection option open for clustering and scalability purposes, but for medium to small scale installations there should be no compelling reason to isolate the database from the rest of the application.
The remaining 25% is tied up in transactions. Databases are aware of transactions insofar as to avail recoverability. Once a transaction completes the data is guaranteed durability, but all records of the transaction are lost. Who instigated the change, when, why, what sort of data was changed? These questions can only be answered if we had made extra provision for the transitional data, by keeping it in a separate set of tables, a flat file, on tape or otherwise.
Speed and transaction-awareness is where the emphasis is at. And this is exactly what DTS/S1 ‘Pitch Black’ is all about. Pitch Black represents all the knowledge about relational and object databases that I have acquired in over 5 years. Its a transaction-oriented database. This piece of software has kept all of the ‘good’ in databases (hardly anything) and has changed all that is ‘bad’ (pretty much everything).
I’ve referred to this Gen 2.0 product a ‘transaction-oriented database’. Unlike everything else out there, everything that Pitch Black does is auditable. Like all other persistent entities, these audits are available to the application as plain old Java objects (POJOs). This changes the playing field completely. You don’t have to buy a transaction processor with mediocre database search and retrieve capability. Nor do you have to buy a mainstream RDBMS or ODBMS and then hand-code the transactional parts. There is now a single product that can do both these things, and can do them well.
We didn’t just go and build Pitch Black to prove a point. Or even to earn some money. I’m the managing director of Obsidian Dynamics and our sister company ventured into financial markets and set out to build a next generation trading platform for Web 2.0. In case you haven’t noticed, 99% Web 2.0 sites aren’t commercial. The whole Web 2.0 umbrella is about openness, building communities, peer interaction. But commercial trade has been a foundation for building communities for centuries. So we decided to amalgamate start-up venture capital and general principles of a securities exchange with the Web 2.0 community. The traditional VC model, just like about anything else, has flaws. In light of the recent economic downturn these flaws are becoming more apparent. The core problem is that a small handfull of people decide whether a new startup should be granted funds. If they’re wrong in their decision, the outcome will amount to either a bad investment and massive financial write off, or a missed opportunity. One solution to this problem is to bring a startup company to the collective attention of millions of Internet users, and let the people decide in a startup’s future by investing their own money.
We have researched this model thoroughly from both a financial and a technical perspective. We came up with the following conclusion. When building a Web 2.0 business, in most cases its fine to slap some PHP on the front, ram a MySQL through the back, and Bob’s your uncle. For a majority of Web 2.0 auditability is a non-requirement. But when you’re taking money from people and issuing them with securities, conventional tooling is insufficient, and you would otherwise be forced into writing a transaction processor on top of a relational database. As you know by now, I’ve had it with databases and I simply wouldn’t stand for it.
Right from the word go, we’ve designed Pitch Black to unify all enterprise data storage requirements into a single, cohesive solution. We have achieved this by closely mimicking the heap allocation model of memory-managed development environments such Java, and depicting similar object-lifecycle semantics in a persistent storage model. In plain terms, what this amounts to is the ability for application developers define classes as they normally would, instantiate those into objects, manipulate various fields and have those changes transparently committed to stable storage. Transactions here are critical. Pitch Black doesn’t simply use the concept of a transaction to populate its REDO log, it keeps transactions indefinitely and exposes them to the application layer when required.
The strength of what we’ve built is that nothing what exists was put in as an afterthought. Every single feature of Pitch Black existed in our minds and our hearts before it was implemented. The resulting product is precisely what we thought it would be and fit with our business model perfectly. But at the end of the day, the trading platform is only half the reason why we built it. Its is a fully featured commercial turnkey solution that anyone can use right off the shelf.
At this point in time its difficult for me to speculate on the future success of Pitch Black. We’re using it internally and we are loving it. But one thing I do know for sure is that we were the first ones there: the first turn a “Database 2.0″ concept into a reality, and no-one’s going to take that away. Being the principal developer of DTS/S1, what thrills me is that if anyone else comes up with an implementation, they will be motivated partially by us and the precedent that we’ve set. We’ve made something that will work for years to come, and this makes me feel liberated. Finally I don’t have to do what I’ve been doing all these years, which is what I’ve HAD to do. Now, I can do what I WANT to do. I can concentrate on building a business app by thinking about it in terms of objects, components, models – going forward and seeing things work rather than worrying about SQL, database licensing, fixing bottlenecks, tuning queries, ORMs and all that’s evil in this world. A transaction-oriented database is here. And we’re going to fight hard to make it stay.
Emil
http://obsidiandynamics.com/
Go!!
As I am writing this post, we are launching our website to give you a little insight as to what gTrade is and how we are planning to revolutionise the Web 2.0 and Venture Capital market.
So what is this thing you call gTrade?
gTrade is a global trading platform provides investors and venture capitalists a platform to trade equity in an open market for emerging and seed Web 2.0 companies.
We represent the next generation securities exchange: not just trading conventional equities, but enabling everyday investors to purchase shares in companies that have not yet listed publicly. As opposed to traditional venture capital, gTrade brings a start-up company to the collective attention of millions of Internet users, letting the people decide in a start-up’s future by investing their own money.
The gTrade marketplace also facilitates the connection of businesses, Venture Capitalists and the average investors. Users trade equity on the online market just like traditional equity markets such as the Nasdaq, Dow Jones of FTSE. This allows you, the average investor to get in on the ground level of the next big Web 2.0 start-up like facebook, Google and the likes.
“So when does the marketplace go live?” I hear you say?
Soon! We are working hard on making the platform as robust as possible. As such we are taking our time to get it right. You can help us out by joining our private beta program on our website (http://www.gtradenet.com) with mock companies and faux money. This is a limited private beta, so we apologise to those that apply and don’t get accepted – its nothing personal.
We will be keeping everyone informed on our progress through this blog – so feel free to subscribe. We will also be posting our thoughts on many topic areas, particularly around Web 2.0, Venture Capital, Financial Markets and Technologies we are using.
We look forward to helping develop a brighter future for the Web 2.0 and emerging markets.
Guy Havenstein
CEO & Co-Founder
Object-oriented databases are naturally fast. Much faster than their SQL counterparts. They are re-emerging in the world of embedded devices, video games, CAD applications – the general class of desktop applications that require persistent storage but don’t necessarily have a large server to connect to. They can get storage from an ODBMS, because they don’t rely on inefficient relational technologies that warrant large and expensive computer hardware.
Now consider a commercial application which operates in a transactional business model. These include banks, the stock exchange, telecommunications, billing systems and so on. Basically anything that relies not only on secure storage of customer data, but also on the storage of all those activities that have led to the present state. Each such activity is a transaction, and these too, need to be stored and retrieved from time to time. Previously, object databases weren’t quite ready, while relational juggernauts weren’t even competing in the transactional arena. Sure, a relational database offers a COMMIT and ROLLBACK, but these serve purely to demarcate a transaction so that it can be recovered should a rat chew through the 30 amp power cord (God have mercy on its soul). No relational database will allow you to query the transaction records and, in fact, there isn’t even a SQL standard that governs this sort of “transitional” data. So if you are a financial institution in need of proper auditability of your customers’ actions, you had to follow one of two paths.
The first path is simple… at first. You buy an Oracle license. Make that 10 licenses because you will need a cluster to make a relational database write records at an acceptable rate. You then plan out a schema, as you would, but allow for a few extra tables that would store transitional information. For instance, if you wanted to store the fact that customer A deposited $10 into customer B’s account, you would need a table with at least 3 columns: 2 for the customers’ account numbers and 1 for the amount, as well as some additional data such as the date/time to log when this transaction had occurred, as well as some unique identifier. Don’t forget to index this, by both account numbers and the transaction ID, otherwise the data is next to useless. Presto! Now you can query this table with SQL and you have yourself a makeshift transaction processor.
The second path is to purchase IBM CICS, or some other dedicated transaction processor that is inherently aware of not just the current state, but also all of the transitional elements that have collectively formulated that state over time. But you don’t really want to do this. CICS is morally outdated and is generally reserved for legacy stuff. The impedance mismatch with object-oriented languages is enormous. Building with a greenfield app with CICS now, would be like building an LCD TV with valves. And, on top of everything, it locks you into a proprietary OS.
So back to the first path then. Now we have to track transitional data, as well as everything else in separate tables. When a transaction is processed, two independent sets of tables are modified. The workload has just doubled. Remember when I mentioned 10 Oracle licenses? Let’s make that 20.
But outright power isn’t everything. This is a well-known fact in motorsport; known but not always appreciated in high performance computing. What about the flexibility of data?
A little while ago I received a statement from my bank, outlining the transactions on my account for the recent month. One transaction that caught my eye was an information message, which on that occasion stated “Your interest rate is now …”. But oddly enough, there was an entry beside this message in the credit column. The amount was $0.00. An innocent entry, no question about it. But it is obvious that the bank’s transaction processor is incapable of adequately persisting transactions of different types, and so the developers have simply shoehorned an information message into a credit transaction. This is the second largest bank in Australia, by the way.
When people piggyback a transaction processor atop of a relational databases, they face the same problems they did before, only now they have to solve it a second time. Just as well, because transactions are too, rich objects with an arbitrary structure that doesn’t fit in a relational model. Personally, I’ve seen solutions that use XML fragments in VARCHAR fields and BLOBs to solve this problem. My bank has solved this problem using the least sensible way: through denormalisation. This really is an example of a square peg in a round hole, and if you happen to be a DBA or a back-end developer, you would have experienced worse examples than this.
So why did I do it?
Because I know that, given a few more years of ignorance on behalf of people in my position, CICS won’t be kicking while Oracle and Sun’s new foster child MySQL will dominate the market. Not in any better form that they are now, just less competition due to the Sun / Oracle duopoly. People will be using relational databases for everything and SQL will be taught in secondary schools. People are already accustomed to ORM frameworks for stateful data, and soon (if not already) this will naturally be applied to transitional data. And so there you have it, a new age transaction processor, sitting on top of a MySQL cluster, running a dozen machines and using O-R mapping frameworks to alleviate the pain of object-oriented transaction processing. Unstructured data is still stored as BLOBs and the whole system barely manages 100 transactions per second. It costs about $1,000,000 to purchase and requires about a room full of cooling equipment.
Emil
http://obsidiandynamics.com/
I wanted to see if I could pull it off. Write a fully transaction-aware native object database that would compete directly with other relational and object databases, and give them all a run for their money. It took months of painstaking work. But all this is behind me now. What lies ahead is even more painstaking work! Except this time I’m not trying to tell a computer what to do, I’m trying to tell the developer community what to do.
I’m not doing it for the money… would have been a misleading title for this blog. I would be lying and you wouldn’t believe me anyway. Oh no, open source is good; when the software is used in the right context. Forums, most Web 2.0 sites, blogs, the word processor that I typed this on. Even small-scale commercial products. But all software fails: open, closed, or slightly ajar. And when you are running a million dollar a day business, and you are hit with unforeseen downtime, you need to act. Proper technical support, timely maintenance releases, superior product quality as a result of a centralised, cohesive developer base. And should the worst occur, someone to with money to blame. Clearly, when your software is in the critical path of an organisation, it needs to be closed, commercial and with a dollar figure attached to it. When we are talking about the storage of peoples’ account balances and transaction records, anything “free” is not an option.
So I did it for the money then?
I started working with relational databases when I was in my late teens or early twenties. Then, it was Oracle 8, DB2 – the usual suspects, and MySQL which was still emerging and, if I recall correctly, could barely handle joins. And then there was Postgres…
Back then we had to develop an app that, in its core, had to manipulate and store data of arbitrary structural complexity. We had Oracle 8. We went relational. The snowball lasted longer in hell. I was then responsible for CRM, while the chief technologist thought of a cunning way of taking the data, ramming all the fixed fields in appropriately typed rows, and all the loosly structured data as a lightweight XML fragment into a large VARCHAR. And it worked! That’s because there was nothing relational about the way we had implemented it. I can’t speculate about the success of this blog, but if it ever makes it beyond this kwrite session and the people who made it work ever get to read this, I just hope they aren’t saying “yep, and we are still doing it that way”.
To our credit we did look into other products. I spent about a month on a feasibility study alone. Comparing independent benchmarks of databases, mainly for outright performance. Then juxtaposing those against the price, features, and anything else that could somehow influence what we would actually license for the production system. Good friends at the time were willing to lend their Oracle 9i staging servers, which means that I can end this sentence now: the outcome of the feasibility study was largely irrelevant. We did look at emerging technology: native XML databases. If only they had been more mature at the time, we would have looked further. Later I had realised that there was a valuable lesson to be learned. Don’t look far for what’s near. We had never even considered object-oriented databases then.
It’s ironic how we berate constructs that try to achieve more than what they were engineered for. Yet seldom do we consider an RDMBS. Why? Because its everywhere, used by everyone and so it must be the right tool for the job. Back in my uni days we were required to develop a SCADA system. The DB was essentially chosen for us: a product by Intersystems. It happened to have been an ODBMS. Although it was object-oriented, I’d rather have worked with Oracle, for this thing was nothing short of evil. Its driver support for Java was just comical, and it led to a lot of silent data corruption. When I went to my lecturer to complain, his response was something like “fool, this thing is really good because people in the field are using it”. He then proceeded to enumerate over some companies who were. Walking away was better than losing marks and so I did just that. If grades weren’t at stake, I would’ve brough up the best counter-argument there is: the QWERTY keyboard. For the uninformed, a QWERTY keyboard is the most inefficient contraption there exists in the solar system. A poor feat of engineering? No, seemingly intelligent at the time it was designed that way because early typewriters would simply jam as the typists got faster. So they “fixed the glitch” by conceiving a keyboard layout most inefficient, with frequently used letters spaced so awkwardly as to affect “flow control” over the typists of the era. Now look at your left hand. You are probably touching it right now. So its settled then. Just because something appears often, it is by no means good.
I brought this up to instill genuine doubt in the reader, that WILL make them a better engineer. And it goes a little something like this. Nothing you see, touch, use or hear about is how it was meant to be. That is the fundamental concept of progress. For if we were content with everything, we would still be cave painting the pretty ape next door. Did I say door? You get the point.
So I did it for the progress then?
Like the QWERTY, an RDBMS was a seemingly intelligent conception at the time. Computers were larger than your apartment and data was flat. Now the world of enterprise information systems has changed completely. Business data is large, complex, ever-changing, but we are… still… painting the ape next door. But to our credit, we are amazingly adaptive. Just like we learned to type fast on keyboard that was designed to type slow on, we have also learned to store complex data in a relational database. Using XML, BLOBs, ORM frameworks, or whatever else you may have thought of. Still, we have failed on one frontier. Seems that whatever we do, we just can’t make that SQL database run fast.
Are we trying to kill a fly with a hammer? Consider how an object is stored in an RDBMS. By virtue of normalisation, the object is “unpacked” into a set of two-dimensional tables. Fragments of the object, potentially objects referenced from within it, are stored as rows in the said tables. Rows in related tables reference each other indirectly, through primary and foreign keys. By now the data is stored. When retrieving the data, we are primarily interested in obtaining the same object that we persisted some time ago. We query the database in such a way as to “repack” the constituent rows back into the object. Oh and the queries themselves can be a work of art. Back in the days of developing a CRM app atop of Oracle 8, I remember writing nested SQL queries 7 levels deep. I was sure of one thing: I never wanted to do that again.
The pitfalls of relational databases are in the excessive overhead required to unpack and then repack data to marshal an object. Consider what really goes on underneath. The packing process has to build a cross-product of tables to denormalise the persisted data. In most cases, this operation is infeasible, and so rows are filtered and query optimisers are employed to establish the most computationally efficient path to combining the candidates. As the data sets grow, so does the query execution time, unless indexes are employed. These help optimise the queries by organising the candidate rows by similar attributes, rather than performing linear searches through the potentially massive data sets. But this impacts storage performance, as indexes take time to build and maintain.
True object databases don’t unpack and pack data, and if they do, its not done quite the way that relational databases do. An ODBMS stores object data verbatim. Then there are post-relational (bitter) flavours, that unpack the data, but keep direct references to constituent parts so that they can be combined quickly. Post-relational DBs are somewhat of a hybrid between ODBMS and RDBMS technologies, but tend to err on the relational side. These are still quite restrictive in terms of data structure, and can be approximated as a relational database with an integrated ORM. Ironically, one of the most prominent P-R DBs, Matisse, markets itself using the phrase “Eliminate Object-Relational Mapping”. Elimination is not a synonym for concealment. Post-relational databases still require a pre-defined schema, but at least they support polymorphism. Perhaps in the same way that Windows 3.11 was called an operating system, post-relational products are called object databases.
Why didn’t object databases evolve to dominate the market? Honestly, I don’t know. Perhaps it was the same reason why the Dvorak layout didn’t displace QWERTY. Twenty times more efficient, but only appealing to about the same number of people. Admittedly, early object databases had one flaw: no support for querying. Well not exactly. You could query a persistent object store by sucking in all the data, and then looping over it in native code: a loop with and a bunch of IF statements. Client-side querying, if you please. Even after this was rectified with the advent of OQL (a la SQL port) object databases still lived for the minority. So if there was nothing wrong with this technology, what was really wrong with it? The answer is: the legacy that relational databases have behind them. Indeed, legacy is the single most overlooked factor when designing and marketing a piece of software. Sometimes when they say “The market is there for X because every man and his dog would want X”, the reality is often “The market is not there for X because every man and his dog have already bought Y”.
Should we then cast object databases into the superior-but-failed basket? Along with Dvorak keyboards, Betamax video tapes and a few others? No, because I haven’t mentioned the words “cost” and “transaction” yet.
Emil
http://obsidiandynamics.com/
Welcome to the inaugural post on the gTrade “Under the Covers” blog.
This aim of this blog is to give you an insight into the management thoughts and updates on the gTrade trading platform. From time to time, gTrade employees will take turns in writing posts for your perusal. These posts will vary from business focused to a technical update.
We encourage you to use the comments section as we really want to know what you think – after all we are making this platform for you!!
Be sure to subscribe to this blog as all major announcements will be made on here first.
– gTrade Management Team
