|
Comments to the fx-devel mailing list.
COPYRIGHT: this document may be freely redistributed in its entirety, including this copyright notice. Redistribution of parts of this document requires the consent of the editor, Ken Kittlitz (kittlitz at ideosphere dot com). All other rights reserved.
Contents
FXTP, the Foresight Exchange Transport Protocol, is a standard that allows for distributed development, scalability, interface independence, minimized data traffic, and platform independence in the implementation of an FX market.
A clearinghouse (C/HS) consists of file server(s) (one or more), mail face(s), web face(s), a connection server, and a single database server (DBS). This design provides for scalability in that the web and mail faces and file servers may be on a separate machine(s) from the DBS. Also, there may be more than one C/HS in the market (for example, if the DBS is still too heavily loaded).
FXTP is what the DBS uses to interact with its web, mail and telnet faces. The DBS is optimized for efficient processing of transactions: it does not prepare hypertext or serve data files, and it does not prepare mail messages (except for transaction confirmations). The DBS process stays resident in memory (RAM and virtual RAM) to eliminate the need for multiple processes (and their loading time) and reduce data loading time (caching keeps the data for more active claims resident).
Each connection to the C/HS (whether via the web face, the mail face, or a direct connection) spawns a connection server process which ensures that a complete (atomic) request has been assembled before opening a conversation with the DBS. All authentication happens within the DBS, the connection server merely forwards complete FXTP requests.
Figure 1: FX objects and their interactions
In the old model for FX, all interaction took place through the DBS, regardless of the bandwidth requirement or latency of the information (latent = static, inverse of real-time). As shown in figure 2, the distributed architecture is intended off-load high latency functions from the DBS to the more appropriate file server or mail face, depending on bandwidth requirements. A major contributor to the bandwidth requirement in FX is the number of people that would be expected to view the information). High bandwidth and low latency applications (multimedia typically, e.g., the hypertext transaction form are also off-loaded by moving them to the web face.
Direct connections to the DBS are limited to low bandwidth access, such as the telnet face or a face on the user's system (e.g., Java).
High +-------------+-------------+
^ | | |
| | File Server | Web Face |
| | | |
Bandwidth +-------------+-------------+
| | | Direct |
| | Mail Face | Connection |
| | | to DBS |
Low +-------------+-------------+
High ----- Latency -----> Low
Figure 2: Application Requirements
This principle is enacted in two ways: filtering and user profiles.
FXTP will allow user customizable interfaces, such as adding a filter to display a transaction form with less than the full complement of claims. In general these interfaces should filter at the server level even at the expense of server processing time. In that case data traffic is minimized if only the information on those claims is sent, rather than sending the entire list and have the client filter it out. The assumption here is that it desirable to minimize the amount of information passed between the server(s) and the client. This a customer service issue (imagine that the customer is on a slow dial-up).
Similarly, user profiles are stored at the server, rather than communicated to it (in detail) from the client at each connection. Support multiple profiles under a single user account, by sending a profile key.
By using the FXTP as defined in this document, users can write applets for their own use or to contribute to the library of FXTP routines.
Users may write unregistered agents that talk to the connection server. Users may also register their applets to run on the C/HS machine(s). Registration of applets allows access by other users for enhanced functionality (i.e., automated unit trading, described elsewhere in this document). To maximize portability between systems and simplify our security screening, only Perl, Java, and C source will be accepted (in order of preference). Executables cannot be registered, and the C/HS owners may refuse to register any source code.
Even though the socket will accept connections from third party mail and web faces, their use is discouraged. There is no way to know if the third party has put in a "trapdoor" to collect passwords. However players may install web or mail faces on their own (presumably trustworthy) systems to improve performance. These remote faces may also be useful for users to test out new applets.
A C/HS operates like a stock brokerage in that a user must have a cash balance at the C/HS, and the C/HS holds the coupons purchased (as with shares at a brokerage). However the C/HS is also the sole repository of claim information such as due date, judge, etc. (whereas a brokerage does not have this responsibility in dealing with shares). Different C/HSs may offer similar or identical claims and arbitrage will keep the prices in step, even though the coupons are not interchangeable.
- regardless of whether the user uses the web face or email face, he will receive some email from the C/HS, such as registration and transaction confirmations.
The connection server sends "atomic requests" to the DBS, that is, it ensures that a complete request is available before it attempts an exclusive (locked) connection to the DBS. This is an efficient way to ensure that the DBS is not occupied by conversations which do not terminate.
[The connection server may also extract data from the file server -- for sake of simplicity, only the DBS is referred to here. Whether the data is available on the file server or must come directly from the DBS is irrelevant to the connecting program, as the connection server determines where to go for the data. In this way, C/HS administrators may move functions between the servers (according to system performance) without affecting the design of applets or faces.]
Whenever a connection is made to the specified socket, whether from the C/HSs own web or mail face or an outside connection, a connection server process is spawned. This process can timeout if a complete request is not received (again, without tying up the DBS waiting for the timeout).
The conversation between the connection server and the DBS need not be in FXTP. However, there may be many copies of the connection server resident in memory waiting for the DBS to service them: the simpler (smaller) they are the better. This means that (for the foreseeable future) the connection server will just relay the FXTP commands, similar to the mail face.
To maximize processing speed, and eliminate the need to have a Perl interpreter in memory for each connection server process that gets spawned, this program is written in C.
Any request scheduling (e.g., giving web face requests precedence over mail face requests) is done at the connection server.
The web face(s) add HTML to the C/HS output (note that this not the same as a web server, thus the difference in name). As far as the user is concerned, the interaction with the web face will be identical to the old FX system: he will not be aware that the web face is opening an FXTP conversation with the C/HS.
Web face processes are spawned as required, so there may be more than one active at a time.
A mail face allows people without a WWW browser to play FX, providing a general interface at the expense of speed and bandwidth. In general, it does little processing of its own, besides reformatting requests in the body of the mail messages into FXTP form.
A mail face's responsibilities are:
- Recognize valid FXTP commands in the body of the message (or explode composite commands) and send these, one at a time, to the Connection Server.
- Maintain a queue of requests, matching connection server responses with the appropriate requests.
- Mail the server responses to the user, after formatting as appropriate.
- Attempt to prevent mail loops by recognizing keywords (e.g., "returned mail") in the message.
- Uuencoded GIFs and other binary data in the connection server's response prior to mailing the response.
The following sections expand on these responsibilities.
Architecture/Operation
The server's responses will be in "raw" FXTP. It is the mailface's responsibility to format the response if necessary before mailing it to the user.
Mail Face Commands
The mail face will support all relevant FXTP commands. Note that some commands, such as an order to buy or sell coupons, will be responded to only with an acknowledgment (e.g., "Your order XXX has been successfully booked"). If and when the trade occurs, the DBS would send an email confirmation.
The telnet face shares much of the code of the mail face.
- It maintains state (as the mail face does), so that the user only needs to enter a uid and password once per session.
- It recognizes the same composite commands as the mail face
Applet is the collective name for customized interfaces and agents that are dependent on the FXTP.
Applets used as customized interfaces can monitor many claims to provide updates on advances in science and technology.
If an applet is executed without user interaction, either at a regularly scheduled interval or as the result of market dynamics, then it is known as an "agent". One such agent regularly updates the reports on the file server by accessing the DBS (through the connection server, of course).
Agents may also make transactions, and are then known as "Trading Agents". Trading Agents increase transaction volumes and awaken hacker interest. They also allow patrons to automatically subsidize trading in a claim in the form of an information prize.
Trading Agents may be registered or unregistered. If a player register a trading agents, it can operate a mutual fund, with the purchase and redemption of the agent units handled automatically at the CH/S.
In many ways Registered Trading Agents (RTAs) operate as a human player, in that they may book orders under the trading agent's name [transaction confirmations for agent to /dev/null ?, or allow a Reply-to address that is different from the name of the person registering]. RTAs do not receive an allowance in the free play aspect. Players buy units in the RTA at the current price, as with mutual funds in a stock market. The RTAs must maintain a cash reserve so that players can cash out at the current market price (i.e., without driving down the value of the other agent investors' holdings). There may be a limit on the amount of a player's holdings that may be cashed out in one day (e.g., or 30% of the RTAs holdings, whichever is less) without a back end penalty being applied.
There is not a similar distinction between registered and unregistered faces because registered faces separate from the mail face and web face don't exist: if the C/HS owners endorse a user's new function it is simply included in the appropriate face. While a RTA may access other C/HSs (to run a mutual fund that invests in various market), faces on one C/HS should not connect to other C/HSs as that would go counter to the idea of distributing the load.
Detailed description of the FXTP commands can be found in the FXTP command set.
This is the server's response to the 'help' command.
user_info uid,[password],param[,param...]
where params are:
uid
gid
email ...requires password
privacy
nym
cash
fees_in
fees_out
allowance
date_signup
date_last_on
holdings ...returns a list
score ...calculated, calls networth
networth ...calculated
book ...calls 'book uid,claims'
list_uids [uid | *]
claim_info (claim regexp),param[,param...]
where params are:
short
keywords
close ...closing date
judge
date_created
long
bid
ask
last
price ...returns list of (bid,ask,last)
status
owner
newsgroup
scaled
book ...calls 'book uid,claims'
list_claims ...calls 'claim_list .*'
claim_list (keyword regexp)
list_keyword
volume [days] ... defaults to last 14 days
plot <claim>[,start][,end][, GIF|ASCII] ... defaults to creation,today,ASCII
(other commands)
account uid,profile_key
book uid,claims ...called by user_info and claim_info
profile uid,key
set_profile uid,password,profile_key,profile_value
password uid,old_pass,new_pass
ticker uid[,claim[,days]] ...should allow * for claims
resend_id email
register email,nym,privacy ...first reg'n step (returns password via email)
activate email,password ...second reg'n step (returns uid)
(session commands)
help
time
bye
(unsupported commands)
user/claim editing
orders uid,password,(new|del|inc|dec),(B|S)Q[N]@price
capitalization
(mail/telnet face FXTP extensions)
login uid,password ...inserts 'uid' and 'password' into
subsequent commands where needed. For example:
login 33,foobar
user_info ,,holdings (== user_info 33,foobar,holdings)
profile ,3 (== profile 33,3)
Notes
user_info: uid = 1 instance, no list of uids
claim_info: CLAIM = a single trading symbol or *. Lists not supported
ticker information: the following commands are planned but not implemented
(start & end (optional) are explicit dates, if used)
capital (uids claim start end)close (uids claim start end)high (uids claim start end)history (uids claim start end)low (uids claim start end)volume (uids claim start end)users_holding (uids claim start end)
Profile Keys
1 to 5 ... user defined (context sensitive)
0 [not used]
-1 to -3 ... system defined
The scheme is open ended (on both ends), that is, users may be allowed to add more of their own keys, and more system defined keys may be added.
The "set_profile" command allows user keys from 0 (things like time zone of the client) through 5 (1..5 are the account form profiles). Each item in the key should be entered on a separate line for mail face (a new multi-line command) so that the individual fields can unambiguously tab delimited. E.g., one profile key element may be 'K+Misc:New Age', so can just split on spaces. Accomplish this by requiring html input, <p> as delimiter which gets mapped into a tab.
The profile keys (1-5) are stored in a user database. The format of these profiles is:
[K+keyword]
[...]
[C+claim]
[...]
[C-claim]
[...]
[...] indicates that multiple entries of the same type are allowed. Entries may appear in any order
When accessing an account report, the profile key is a number or keyword regexp.
Other system defined keys can return the list of claims a user either has coupons or booked orders in. In an open game, this is how to get a terse account form that shows what another user is up to.
The following system defined keys have predefined functions, but generate different claims lists depending on the user id they are applied to (thus they are context sensitive). Specifically:
-1 claims with booked orders
-2 claims with holdings
-3 claims with booked orders or holdings
Not all (future) system defined keys need be context sensitive.
Since the profile key zero is not used above, that is where the more generic user profile is stored. Specifically, it is a string of:
[-12 to 12] timezone
[0 or 1] whether to show prices for NO coupons
[0 or 1] whether to show short descriptions
[integer] length of the order box
[0 or 1] whether the user's client supports multiple submit pushbuttons
[0 or 1] whether the user has a graphical browser (e.g., plots in GIF or ASCII)
Nyms
Nick Szabo <http://www.digicash.com/~nick/smart.contracts.glossary.html> writes (reprinted with permission):
Nym: An identifier that links only a small amount of related information about a person, usually that information deemed by the nym holder to be relevant to a particular organization or community. Examples of nyms include electronic bulletin board nicknames, pen names, aliases, and brand names. A nym may gain reputation within its community. For example, a conglomerate may sell a wide variety of brand names, each reputable in its own market niche. With Chaumian credentials, a nym can take advantage of the positive credentials of the holder's other nyms, as provably linked by the is-a-person credential.
Nyms don't change (privacy flag can't, but email can). There is a privacy flag, that, if set to zero ("off"), includes your email address in the nym. Default is off, i.e., let other players know who you are.
Several people have contributed to this design:
![[Previous]](/fx/images/PREVBUT.GIF) ![[Top]](/fx/images/TOPBUT.GIF) ![[Next]](/fx/images/NEXTBUT.GIF)
| 
