Re: [PATCH] (super)user-loadable mysqld parsers

Attached is a dynParser patch to add alternate parsers into mysqld (and a parser that uses this interface). I developed this for the SPASQL project [SPA], but expect folks might want to use it to add parsers for other variants of SQL as well as totally different languages like XQuery and SPARQL.
  [SPA] http://esw.w3.org/topic/SPASQL

The extension works by adding three new request types to the client/server protocol: PARSER1, PARSER2, PARSER3. The API function mysql_send_query takes a fourth parameter in order to send this to the server. On the server side, sql_parse.cc responds to requests for PARSER{1,2,3} by invoking dynamic_parsers[N].initialize->parse(). See [SRC] and [DYN] (below) for compilation and execution instructions.   [SRC] http://www.w3.org/2005/05/22-SPARQL-MySQL/#srcbuild   [DYN] http://www.w3.org/2005/05/22-SPARQL-MySQL/#dynParser

A couple notes about protocol compatibility: a stock 5.x client does everything we need except invoke PARSERn requests. This is 'cause I left COM_QUERY where it was, and stuck the user-selected parsers at the end of the enum.
  /* add-on parsers */
  COM_PARSER0, /* !!! change to COM_QUERY on next protocol change */   COM_PARSER1, COM_PARSER2, COM_PARSER3, The cost is a slight discontinuity in the code for COM_QUERY vs. the other parsers. On the other hand, I haven't made MYSQLparse conform to the user parser API so it requires special code anyways. (I never figured out how to set defaults in set_var so I couldn't gracefully make MYSQLparse the default parser, plus it's dangerous to have the core parser be a shared library, and finally, if you did change parser0 from MYSQLparse to some other parser, you could find yourself with no tested way to talk to mysqld.)

COM_PARSER0 is only used as a base for counting parser offsetsd; it is never passed so we could save an enum by assigning it to the previous value:

  /* add-on parsers */
  COM_PARSER0 = COM_STMT_FETCH,
  COM_PARSER1, COM_PARSER2, COM_PARSER3, but it may make sense to move COM_QUERY there instead.

Brian and I discussed implementing a stacked parser approach:   mysql> add_parser("libsparql", ParserPos_FRONT);   mysql> SELECT ?s WHERE { ?s <foo.bar> "hibbyhop"; The cost of this is that you can't force any particular parser to handle a particular request (unless you remove_parser and add_parser to get the one you like up front for that query). It would, however, provide a neat extensibility mechanism where 3rd party providers could supply parsers for add-on functionality. I prototyped this a bit in a Dynamic_parser and believe that a Dynamic_parser could provide a parser that stacked other Dynamic_parsers and even target queries at a particular parser if one chose. However, I have not really implemented this.

Any feedback will be enthusiastically welcomed. I'm motivated to type madly on this on the off-chance I can get it into the 5.2 release as I have several pharmaceuticals interested in the SPASQL project and am anxious to get them working with the MySQL implementation.

-- 
-eric

office: +1.617.258.5741 NE43-344, MIT, Cambridge, MA 02144 USA
mobile: +1.617.599.3509

(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related lists.mysql.com Links announce benchmarks bugs certification cluster commits dotnet gui-tools internals java maxdb myodbc mysql mysqldoc packagers perl plusplus replication win32 Request more information about Pantek