Re: Re: [PATCH] (super)user-loadable mysqld parsers

• Arnold Daniels <info@adaniels.nl> [2007-09-05 20:44-0400]
  > Hi Eric,
  >
  > First of all let me say that at least one person is enthusiastic (me). I do
  > have a few question remarks:
  >
  > You mentioned about the possibility to add an XQuery parser. But then how
  > would you map the data in the storage engines, which is relational data, to
  > XQuery? Or are you thinking about a special XML engine? Personally I like
  > the way DB2 solves this. You do an pass a value to the XQuery
  

> SELECT XMLCAST(XMLQUERY(for $i in $t/product/description/category return
> $i' PASSING BY REF T.DESCRIPTION AS "t" RETURNING SEQUENCE) AS
> VARCHAR(128)) FROM xmlproduct AS t

The XQuery portion of your query was
  for $i in $t/product/description/category return $i which leads me to guess a schema where the XPaths represent joins through tables ala:
  SELECT XMLrepresentationOf(something.primaryKey)     FROM something
    JOIN Products ON something.product=Products.primaryKey     JOIN Descriptions ON Product.description=Descriptions.primaryKey     JOIN Categories ON Description.Category=Categories.primaryKey Lots of folks have done XQuery/relational mappings, and I have the impression they have this sort of mapping. This problem can be viewed as:
  1 How would you represent your relational data in XQuery?   2 How would you query that representation? Given that approach, you need knowledge of the schema rather than a lot SQL-like code mapping the XML names and type to SQL terms. That, however, is if you wish to do everything in XQuery. If you want the outer language to be SQL, then yes, something like the DB2 name/type mapping is needed.

> This functionality cannot be mimicked by a function, like ExtractValue(),
> because a function can't make the query return more rows than selected.
> Lets say that table had 7 rows with 3 products in the XML per row, the
> query should return 21 record.

Agreed. Violating these restriction semantics would mean you weren't acutally using XQuery, just something that looked like it.

Aside {
When we started work on SPARQL, we had to prove to ourselves that we shouldn't use an existing W3C spec, namely XQuery. Some approaches were suggested and the ones that violated the predicate-as-restriction rule were dismissed. The remaining ones mostly focused around a canonical representation in XML. We didnt't find those close enough to our data model (and consequently they were too verbose) for our liking. That said, I'm always keen to enable interoperability between query systems. }

> Here you see however that the XQuery parser isn't choosen to be `the`
> parser because this would be quite useless. Instead the XMLQuery if the
> directive.

Useless to folks who want to use SQL as their main language, but that's not how the XQuery (or SPARQL) folks see the world. If you offer someone a standard language, it's most useful to them if they don't have to do any transformations into another language. Otherwise, the "standard" query is different on MySQL and on Oracle, or DB2 ...

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related lists.mysql.com Links announce benchmarks bugs certification cluster commits dotnet gui-tools internals java maxdb myodbc mysql mysqldoc packagers perl plusplus replication win32 Request more information about Pantek

> I don't know much about SPARQL, but from what I've read the query does map
> well to relational data. However simply choosing the parser based on the
> syntax seems rather confusing. For instance the '?' is also used to insert
> data from a prepared statement. Now when you forget to type a comma, the
> query is suddenly interpreted as SPASQL. Making a change to the
> client/server protocol doesn't seem like the best solution to me as well,
> because that well have a huge impact across the software development world,
> and not in a good way. A simple solution seems to be to just specify it in
> the query:
>
> SPASQL SELECT ?s WHERE { ?s <foo.bar> "hibbyhop";

That's what I originally implemented, but Brian Miezejewski's proposal was to differentiate parsers based on the protocol. I was fearful of the work it involved, but looking now, the patch actually isn't so large.

> Both cases require that the parser actually integrates with the current
> (SQL) parser, this is probably a bit harder to implement. Also, this seems
> like a perfect candidate for the INSTALL PLUGIN syntax.

I can imagine a few ways to do this:

• Parser Integration (extend sql_yacc.yy and sql_lex.cc): Integrating with the SQL parser has a pretty high implementation and deployment cost. There are currently 285 shift/reduce conflicts in the grammar and adding a language like XQuery, which has 5 special cases in the lexer, would make quiet a soup. I'm doing some work with mechanical grammar recombination in the yacker, but that's kinda researchy. Another drawback is the likelyhood of collisions on popular idioms like set foo="bar".
• Parser Encapsulation (add parser state to dispatch other language): The lexer could have a special state where it expected another language span and had some lexer rules controlling when that span would end. For example: posn | start | end |example start|!(SELECT|SET|SHOW|...)|rest of query|XQuery for $i ... any |CALLOUT(%language, ...|balanced ')' |... IN CALLOUT('XQuery', for $i ...

These require query-time transformation of an otherwise standard language; nice if you see the world through SQL eyes, less so if you see the world through XQuery or SPARQL eyes and want platform-independent queries.

Also, this requires that all the THD manipulation done so far when parsing be compatible with whatever the next parser will do to it. A relational API that composed query plan would be a pretty cool way to handle this sort of integration, but that seems a good ways out. (I'd have a lot of fun working on that, though.)

• Callout function (encapsulate query as a string): This looks like the ... IN CALLOUT('XQuery', for $i ... above, except you need quotes around, and quote escaping of, the foreign query. This is compatible with the User Defined Function interface. It is simple to implement, but hard to use; same challenges as above.
• Prefix matching (look for string before calling SQL parser): This is how I implemented the first SPARQL parser: if (!strncmp(inBuf, "SPARQL:", 7)) { lex->ptr = (uchar*)inBuf+7; sparqlFrob frob(thd, (char**)&lex->ptr); parse_res = frob.parse(); } else { parse_res = MYSQLparse((void *)thd); }

It worked well enough, but required a priori knowledge of all the possible extension languages and the requirement that they not conflict with any possible first token in the MySQL SQL grammar. As described above, this was not a verbatim SPARQL query.

Ulimately, I like the protocol-switched approach as it gives the user explicit control and means that SPARQL queries on MySQL will be like SPARQL queries in Jena or any other implementation.

Do you need more help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related lists.mysql.com Links announce benchmarks bugs certification cluster commits dotnet gui-tools internals java maxdb myodbc mysql mysqldoc packagers perl plusplus replication win32 Request more information about Pantek

==

> Best regards,
> Arnold
>
>
> Eric Prud'hommeaux wrote:
>> Attached is a dynParser patch to add alternate parsers into mysqld
>> (and a parser that uses this interface). I developed this for the
>> SPASQL project [SPA], but expect folks might want to use it to add
>> parsers for other variants of SQL as well as totally different
>> languages like XQuery and SPARQL.
>> [SPA] http://esw.w3.org/topic/SPASQL
>>
>> The extension works by adding three new request types to the
>> client/server protocol: PARSER1, PARSER2, PARSER3. The API function
>> mysql_send_query takes a fourth parameter in order to send this to the
>> server. On the server side, sql_parse.cc responds to requests for
>> PARSER{1,2,3} by invoking dynamic_parsers[N].initialize->parse().
>> See [SRC] and [DYN] (below) for compilation and execution instructions.
>> [SRC] http://www.w3.org/2005/05/22-SPARQL-MySQL/#srcbuild
>> [DYN] http://www.w3.org/2005/05/22-SPARQL-MySQL/#dynParser
>>
>>
>> A couple notes about protocol compatibility: a stock 5.x client does
>> everything we need except invoke PARSERn requests. This is 'cause I
>> left COM_QUERY where it was, and stuck the user-selected parsers at
>> the end of the enum.
>> /* add-on parsers */
>> COM_PARSER0, /* !!! change to COM_QUERY on next protocol change */
>> COM_PARSER1, COM_PARSER2, COM_PARSER3,
>> The cost is a slight discontinuity in the code for COM_QUERY vs. the
>> other parsers. On the other hand, I haven't made MYSQLparse conform to
>> the user parser API so it requires special code anyways. (I never
>> figured out how to set defaults in set_var so I couldn't gracefully
>> make MYSQLparse the default parser, plus it's dangerous to have the
>> core parser be a shared library, and finally, if you did change
>> parser0 from MYSQLparse to some other parser, you could find yourself
>> with no tested way to talk to mysqld.)
>>
>> COM_PARSER0 is only used as a base for counting parser offsetsd; it is
>> never passed so we could save an enum by assigning it to the previous
>> value:
>>
>> /* add-on parsers */
>> COM_PARSER0 = COM_STMT_FETCH, COM_PARSER1, COM_PARSER2, COM_PARSER3,
>> but it may make sense to move COM_QUERY there instead.
>>
>> Brian and I discussed implementing a stacked parser approach:
>> mysql> add_parser("libsparql", ParserPos_FRONT);
>> mysql> SELECT ?s WHERE { ?s <foo.bar> "hibbyhop";
>> The cost of this is that you can't force any particular parser to
>> handle a particular request (unless you remove_parser and add_parser
>> to get the one you like up front for that query). It would, however,
>> provide a neat extensibility mechanism where 3rd party providers could
>> supply parsers for add-on functionality. I prototyped this a bit in a
>> Dynamic_parser and believe that a Dynamic_parser could provide a
>> parser that stacked other Dynamic_parsers and even target queries at a
>> particular parser if one chose. However, I have not really implemented
>> this.
>>
>> Any feedback will be enthusiastically welcomed. I'm motivated to type
>> madly on this on the off-chance I can get it into the 5.2 release as
>> I have several pharmaceuticals interested in the SPASQL project and
>> am anxious to get them working with the MySQL implementation.
>> ------------------------------------------------------------------------
>>
>> diff -au sql/set_var.cc.orig sql/set_var.cc
>> --- sql/set_var.cc.orig 2007-09-03 19:17:28.000000000 -0400
>> +++ sql/set_var.cc 2007-09-04 17:48:12.000000000 -0400
>> @@ -69,6 +69,10 @@
>> extern ulong ndb_report_thresh_binlog_mem_usage;
>> #endif
>> +/* dynamically-loaded parsers */
>> +#include "dyn_parser.h"
>> +#include <dlfcn.h>
>> +
>> extern CHARSET_INFO *character_set_filesystem;
>> @@ -88,6 +92,8 @@
>> delay_key_write_type_names, NULL
>> };
>> +struct dynamic_parser_info dynamic_parsers[3];
>> +
>> static int sys_check_ftb_syntax(THD *thd, set_var *var);
>> static bool sys_update_ftb_syntax(THD *thd, set_var * var);
>> static void sys_default_ftb_syntax(THD *thd, enum_var_type type);
>> @@ -117,6 +123,11 @@
>> static void fix_max_relay_log_size(THD *thd, enum_var_type type);
>> static void fix_max_connections(THD *thd, enum_var_type type);
>> static int check_max_delayed_threads(THD *thd, set_var *var);
>> +static int sys_check_parser(THD *thd, set_var *var);
>> +static bool sys_update_parserN(THD *thd, set_var * var, uchar parser);
>> +static bool sys_update_parser0(THD *thd, set_var * var);
>> +static bool sys_update_parser1(THD *thd, set_var * var);
>> +static bool sys_update_parser2(THD *thd, set_var * var);
>> static void fix_thd_mem_root(THD *thd, enum_var_type type);
>> static void fix_trans_mem_root(THD *thd, enum_var_type type);
>> static void fix_server_id(THD *thd, enum_var_type type);
>> @@ -348,6 +359,21 @@
>>
>> &SV::optimizer_prune_level);
>> static sys_var_thd_ulong sys_optimizer_search_depth(&vars,
>> "optimizer_search_depth",
>>
>> &SV::optimizer_search_depth);
>> +static sys_var_str sys_dynamic_parser0(&vars,
>> "dynamic_parser1",
>> + sys_check_parser,
>> + sys_update_parser0,
>> + 0,
>> + dynamic_parsers[0].name);
>> +static sys_var_str sys_dynamic_parser1(&vars,
>> "dynamic_parser2",
>> + sys_check_parser,
>> + sys_update_parser1,
>> + 0,
>> + dynamic_parsers[1].name);
>> +static sys_var_str sys_dynamic_parser2(&vars,
>> "dynamic_parser3",
>> + sys_check_parser,
>> + sys_update_parser2,
>> + 0,
>> + dynamic_parsers[2].name);
>> const char *optimizer_use_mrr_names[] = {"auto", "force", "disable",
>> NullS};
>> TYPELIB optimizer_use_mrr_typelib= {
>> @@ -828,6 +854,44 @@
>> sizeof(ft_boolean_syntax)-1);
>> }
>> +static int sys_check_parser(THD *thd, set_var *var)
>> +{
>> + if (thd->security_ctx->master_access & SUPER_ACL) {
>> + void* lib = dlopen(var->value->str_value.ptr(), RTLD_NOW);
>> + if (lib) {
>> + dlclose(lib);
>> + return 0;
>> + }
>> + else
>> + {
>> + char* errstr = dlerror();
>> + my_error(ER_UNKNOWN_CHARACTER_SET, MYF(0), errstr); // !!! need new
>> error
>> + return 1;
>> + }
>> + }
>> + else
>> + {
>> + my_error(ER_SPECIFIC_ACCESS_DENIED_ERROR, MYF(0), "SUPER");
>> + return 1;
>> + }
>> +}
>> +
>> +static bool sys_update_parserN(THD *thd, set_var * var, uchar parser)
>> +{
>> + strmake(dynamic_parsers[parser].name, var->value->str_value.c_ptr(),
>> + sizeof(dynamic_parsers[parser].name)-1);
>> + if (dynamic_parsers[parser].handle)
>> + dlclose(dynamic_parsers[parser].handle);
>> + dynamic_parsers[parser].handle = dlopen(dynamic_parsers[parser].name,
>> RTLD_NOW);
>> + dynamic_parsers[parser].initialize =
>> (t_initialize*)dlsym(dynamic_parsers[parser].handle,
>> "_Z10initializeP3THDPPc");
>> + return 0;
>> +}
>> +static bool sys_update_parser0(THD *thd, set_var * var)
>> +{ return sys_update_parserN(thd, var, 0); }
>> +static bool sys_update_parser1(THD *thd, set_var * var)
>> +{ return sys_update_parserN(thd, var, 1); }
>> +static bool sys_update_parser2(THD *thd, set_var * var)
>> +{ return sys_update_parserN(thd, var, 2); }
>> /*
>> If one sets the LOW_PRIORIY UPDATES flag, we also must change the
>> diff -au client/mysqltest.c.orig client/mysqltest.c
>> --- client/mysqltest.c.orig 2007-09-03 19:07:45.000000000 -0400
>> +++ client/mysqltest.c 2007-09-03 14:51:56.000000000 -0400
>> @@ -500,7 +500,7 @@
>> struct st_connection *cn= (struct st_connection*)arg;
>> mysql_thread_init();
>> - VOID(mysql_send_query(&cn->mysql, cn->cur_query, cn->cur_query_len));
>> + VOID(mysql_send_query(&cn->mysql, cn->cur_query, cn->cur_query_len,
>> 0));
>> mysql_thread_end();
>> pthread_mutex_lock(&cn->mutex);
>> @@ -517,7 +517,7 @@
>> pthread_t tid;
>> if (flags & QUERY_REAP_FLAG)
>> - return mysql_send_query(&cn->mysql, q, q_len);
>> + return mysql_send_query(&cn->mysql, q, q_len, 0);
>> if (pthread_mutex_init(&cn->mutex, NULL) ||
>> pthread_cond_init(&cn->cond, NULL))
>> @@ -534,7 +534,7 @@
>> #else /*EMBEDDED_LIBRARY*/
>> -#define do_send_query(cn,q,q_len,flags) mysql_send_query(&cn->mysql, q,
>> q_len)
>> +#define do_send_query(cn,q,q_len,flags) mysql_send_query(&cn->mysql, q,
>> q_len, 0)
>> #endif /*EMBEDDED_LIBRARY*/
>> diff -au client/mysql.cc.orig client/mysql.cc
>> --- client/mysql.cc.orig 2007-09-03 19:07:45.000000000 -0400
>> +++ client/mysql.cc 2007-09-03 14:52:29.000000000 -0400
>> @@ -1828,7 +1828,17 @@
>> for (uint retry=0;; retry++)
>> {
>> int error;
>> - if (!mysql_real_query(&mysql,buf,length))
>> + uchar parser = 0;
>> + if (!strncmp(buf, "parser", 6) && buf[6] != 0 && buf[7] == ':') {
>> + parser = buf[6]-'0';
>> + if (parser > 0 || parser < PARSERS) {
>> + buf += 8;
>> + length -= 8;
>> + } else {
>> + // !!! error: invalid parser
>> + }
>> + }
>> + if (!mysql_parser_query(&mysql,buf,length,parser))
>> return 0;
>> error= put_error(&mysql);
>> if (mysql_errno(&mysql) != CR_SERVER_GONE_ERROR || retry > 1 ||
>> diff -au sql/sql_parse.cc.orig sql/sql_parse.cc
>> --- sql/sql_parse.cc.orig 2007-08-26 04:36:04.000000000 -0400
>> +++ sql/sql_parse.cc 2007-09-05 09:54:19.000000000 -0400
>> @@ -28,6 +28,10 @@
>> #include "events.h"
>> #include "sql_trigger.h"
>> +/* dynamically-loaded parsers */
>> +#include "dyn_parser.h"
>> +#include <dlfcn.h>
>> +
>> /**
>> @defgroup Runtime_Environment Runtime Environment
>> @{
>> @@ -80,6 +84,10 @@
>> { C_STRING_WITH_LEN("Set option") },
>> { C_STRING_WITH_LEN("Fetch") },
>> { C_STRING_WITH_LEN("Daemon") },
>> + { C_STRING_WITH_LEN("Parser 0") },
>> + { C_STRING_WITH_LEN("Parser 1") },
>> + { C_STRING_WITH_LEN("Parser 2") },
>> + { C_STRING_WITH_LEN("Parser 3") },
>> { C_STRING_WITH_LEN("Error") } // Last command number
>> };
>> @@ -431,7 +439,7 @@
>> */
>> thd->query_id=next_query_id();
>> thd->set_time();
>> - mysql_parse(thd, thd->query, length, & found_semicolon);
>> + mysql_parse(thd, thd->query, length, & found_semicolon, 0);
>> close_thread_tables(thd); // Free tables
>> if (thd->is_fatal_error)
>> @@ -842,6 +850,9 @@
>> break;
>> }
>> case COM_QUERY:
>> + case COM_PARSER1:
>> + case COM_PARSER2:
>> + case COM_PARSER3:
>> {
>> if (alloc_query(thd, packet, packet_length))
>> break; // fatal error is set
>> @@ -856,7 +867,8 @@
>> if (!(specialflag & SPECIAL_NO_PRIOR))
>> my_pthread_setprio(pthread_self(),QUERY_PRIOR);
>> - mysql_parse(thd, thd->query, thd->query_length, & found_semicolon);
>> + int parser = command == COM_QUERY ? 0 : command - COM_PARSER0;
>> + mysql_parse(thd, thd->query, thd->query_length, & found_semicolon,
>> parser);
>> while (!thd->killed && found_semicolon && !thd->net.report_error)
>> {
>> @@ -885,7 +897,7 @@
>> thd->set_time(); /* Reset the query start time. */
>> /* TODO: set thd->lex->sql_command to SQLCOM_END here */
>> VOID(pthread_mutex_unlock(&LOCK_thread_count));
>> - mysql_parse(thd, next_packet, length, & found_semicolon);
>> + mysql_parse(thd, next_packet, length, & found_semicolon, parser);
>> }
>> if (!(specialflag & SPECIAL_NO_PRIOR))
>> @@ -5302,7 +5314,7 @@
>> */
>> void mysql_parse(THD *thd, const char *inBuf, uint length,
>> - const char ** found_semicolon)
>> + const char ** found_semicolon, uchar parser)
>> {
>> DBUG_ENTER("mysql_parse");
>> @@ -5336,7 +5348,24 @@
>> Lex_input_stream lip(thd, inBuf, length);
>> - bool err= parse_sql(thd, &lip, NULL);
>> + bool err;
>> + if (parser == 0) {
>> + err= parse_sql(thd, &lip, NULL);
>> + } else {
>> + struct dynamic_parser_info* p = &dynamic_parsers[parser-1];
>> + if (p->initialize) {
>> + char* space = (char*)malloc(strlen(lip.get_ptr())+1);
>> + strcpy(space, lip.get_ptr());
>> +
>> + Dynamic_parser* parser = p->initialize(thd, &space);
>> + err= parser->parse();
>> + delete parser;
>> + } else {
>> + my_error(ER_WRONG_DB_NAME ,MYF(0), "parser not loaded");//!!!need error
>> + err= 1;
>> + }
>> + }
>> +
>> *found_semicolon= lip.found_semicolon;
>> if (!err)
>> @@ -7160,6 +7189,14 @@
>> return err_status;
>> }
>> +
>> +/* fill Dynamic_parser's vtable with dummy functions */
>> +int Dynamic_parser::parse() {
>> +}
>> +
>> +Dynamic_parser::~Dynamic_parser() {
>> +}
>> +
>> /**
>> @} (end of group Runtime_Environment)
>> */
>> diff -au sql-common/client.c.orig sql-common/client.c
>> --- sql-common/client.c.orig 2007-09-03 19:07:45.000000000 -0400
>> +++ sql-common/client.c 2007-09-05 07:54:48.000000000 -0400
>> @@ -2757,7 +2757,8 @@
>> */
>> int STDCALL
>> -mysql_send_query(MYSQL* mysql, const char* query, ulong length)
>> +mysql_send_query(MYSQL* mysql, const char* query, + ulong length,
>> unsigned char parser)
>> {
>> DBUG_ENTER("mysql_send_query");
>> DBUG_PRINT("enter",("rpl_parse: %d rpl_pivot: %d",
>> @@ -2777,7 +2778,8 @@
>> mysql->last_used_con = mysql;
>> #endif
>> - DBUG_RETURN(simple_command(mysql, COM_QUERY, (uchar*) query, length,
>> 1));
>> + enum enum_server_command command = parser ? COM_PARSER0 + parser :
>> COM_QUERY;
>> + DBUG_RETURN(simple_command(mysql, command, (uchar*) query, length, 1));
>> }
>> @@ -2788,7 +2790,20 @@
>> DBUG_PRINT("enter",("handle: 0x%lx", (long) mysql));
>> DBUG_PRINT("query",("Query = '%-.4096s'",query));
>> - if (mysql_send_query(mysql,query,length))
>> + if (mysql_send_query(mysql,query,length,0))
>> + DBUG_RETURN(1);
>> + DBUG_RETURN((int) (*mysql->methods->read_query_result)(mysql));
>> +}
>> +
>> +
>> +int STDCALL
>> +mysql_parser_query(MYSQL *mysql, const char *query, ulong length, uchar
>> parser)
>> +{
>> + DBUG_ENTER("mysql_real_query");
>> + DBUG_PRINT("enter",("handle: 0x%lx", (long) mysql));
>> + DBUG_PRINT("query",("Query = '%-.4096s'",query));
>> +
>> + if (mysql_send_query(mysql,query,length,parser))
>> DBUG_RETURN(1);
>> DBUG_RETURN((int) (*mysql->methods->read_query_result)(mysql));
>> }
>> diff -au sql/slave.cc.orig sql/slave.cc
>> diff -au sql/log_event.cc.orig sql/log_event.cc
>> --- sql/log_event.cc.orig 2007-09-03 19:07:45.000000000 -0400
>> +++ sql/log_event.cc 2007-09-03 14:53:12.000000000 -0400
>> @@ -2083,7 +2083,7 @@
>> /* Execute the query (note that we bypass
>> dispatch_command()) */
>> const char* found_semicolon= NULL;
>> - mysql_parse(thd, thd->query, thd->query_length, &found_semicolon);
>> + mysql_parse(thd, thd->query, thd->query_length, &found_semicolon,
>> 0);
>> log_slow_statement(thd);
>> }
>> else
>> diff -au sql/mysql_priv.h.orig sql/mysql_priv.h
>> --- sql/mysql_priv.h.orig 2007-08-26 18:38:30.000000000 -0400
>> +++ sql/mysql_priv.h 2007-09-03 14:53:51.000000000 -0400
>> @@ -970,7 +970,7 @@
>> bool force_switch);
>> void mysql_parse(THD *thd, const char *inBuf, uint length,
>> - const char ** semicolon);
>> + const char ** semicolon, uchar parser);
>> bool mysql_test_parse_for_slave(THD *thd,char *inBuf,uint length);
>> bool is_update_query(enum enum_sql_command command);
>> diff -au include/mysql_com.h.orig include/mysql_com.h
>> --- include/mysql_com.h.orig 2007-08-25 15:13:27.000000000 -0400
>> +++ include/mysql_com.h 2007-09-03 14:54:03.000000000 -0400
>> @@ -60,6 +60,7 @@
>> servers won't be able to handle them as 'unsupported'.
>> */
>> +#define PARSERS 3 // c.f. COM_PARSER1, COM_PARSER2, COM_PARSER3 below
>> enum enum_server_command
>> {
>> COM_SLEEP, COM_QUIT, COM_INIT_DB, COM_QUERY, COM_FIELD_LIST,
>> @@ -69,6 +70,11 @@
>> COM_TABLE_DUMP, COM_CONNECT_OUT, COM_REGISTER_SLAVE,
>> COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA,
>> COM_STMT_CLOSE,
>> COM_STMT_RESET, COM_SET_OPTION, COM_STMT_FETCH, COM_DAEMON,
>> +
>> + /* add-on parsers */
>> + COM_PARSER0, /* !!! change to COM_QUERY on next protocol change */
>> + COM_PARSER1, COM_PARSER2, COM_PARSER3, +
>> /* don't forget to update const char *command_name[] in sql_parse.cc */
>> /* Must be last */
>> diff -au include/mysql.h.orig include/mysql.h
>> --- include/mysql.h.orig 2007-09-03 19:07:45.000000000 -0400
>> +++ include/mysql.h 2007-09-03 14:54:03.000000000 -0400
>> @@ -446,9 +446,13 @@
>> int STDCALL mysql_select_db(MYSQL *mysql, const char *db);
>> int STDCALL mysql_query(MYSQL *mysql, const char *q);
>> int STDCALL mysql_send_query(MYSQL *mysql, const char *q,
>> - unsigned long length);
>> + unsigned long length, + unsigned char parser);
>> int STDCALL mysql_real_query(MYSQL *mysql, const char *q,
>> - unsigned long length);
>> + unsigned long length);
>> +int STDCALL mysql_parser_query(MYSQL *mysql, const char *q,
>> + unsigned long length,
>> + unsigned char parser);
>> MYSQL_RES * STDCALL mysql_store_result(MYSQL *mysql);
>> MYSQL_RES * STDCALL mysql_use_result(MYSQL *mysql);
>> diff -au include/dyn_parser.h.orig include/dyn_parser.h
>> --- include/dyn_parser.h.orig 2007-09-03 19:10:02.000000000 -0400
>> +++ include/dyn_parser.h 2007-09-03 19:09:52.000000000 -0400
>> @@ -0,0 +1,33 @@
>> +#ifndef _dyn_parser_h
>> +#define _dyn_parser_h
>> +/*-- use only from C++
>> + #ifdef __cplusplus
>> + extern "C" {
>> + #endif
>> +*/
>> +
>> +class Dynamic_parser {
>> +
>> +public:
>> + Dynamic_parser(THD *thd, char** ptr) {
>> + this->thd= thd;
>> + }
>> + virtual ~Dynamic_parser();
>> + virtual int parse();
>> +
>> + THD *thd;
>> +};
>> +extern "C" {
>> + typedef Dynamic_parser* t_initialize(THD *, char **);
>> +}
>> +
>> +struct dynamic_parser_info {
>> + char name[FN_REFLEN];
>> + void* handle;
>> + t_initialize* initialize;
>> + // Dynamic_parser* (*initialize)(THD *, char **);
>> +};
>> +extern struct dynamic_parser_info dynamic_parsers[];
>> +
>> +#endif /* _dyn_parser_h */
>> +
>> diff -au sql/backup/meta_backup.cc.orig sql/backup/meta_backup.cc
>> --- sql/backup/meta_backup.cc.orig 2007-09-03 21:55:06.000000000 -0400
>> +++ sql/backup/meta_backup.cc 2007-09-03 21:55:43.000000000 -0400
>> @@ -361,7 +361,7 @@
>> pthread_mutex_unlock(&::LOCK_thread_count);
>> const char *ptr;
>> - ::mysql_parse(thd,thd->query,thd->query_length,&ptr);
>> + ::mysql_parse(thd,thd->query,thd->query_length,&ptr,0);
>> thd->net.vio= save_vio;
>> diff -au sql/ha_ndbcluster_binlog.cc.orig sql/ha_ndbcluster_binlog.cc
>> --- sql/ha_ndbcluster_binlog.cc.orig 2007-09-03 21:56:23.000000000 -0400
>> +++ sql/ha_ndbcluster_binlog.cc 2007-09-03 21:56:28.000000000 -0400
>> @@ -277,7 +277,7 @@
>> thd->options&= ~OPTION_BIN_LOG;
>> DBUG_PRINT("query", ("%s", thd->query));
>> - mysql_parse(thd, thd->query, thd->query_length, &found_semicolon);
>> + mysql_parse(thd, thd->query, thd->query_length, &found_semicolon, 0);
>> if (no_print_error && thd->query_error)
>> {
>> ------------------------------------------------------------------------
>>
>> %{
>> #define YY_sparqlParser_STYPE yy_sparqlParser_stype
>> /* $Id: sparqlParser.yy,v 1.9 2007/09/05 18:40:30 eric Exp $
>>
>> TODO:
>> research pre-existing var.jt in conjunction with trump
>>
>> OUTLINE:
>>
>> Some SPARQL graph pattern opperations correspond to SQL subselects. These
>> create
>> a new BindingContext, which keeps track of looks for table alias re-use
>> opportunities. These occur in a given binding context where more than one
>> subject/predicate pair is in the same table. For instance, both (?s,
>> T1.p1) and
>> (?s, T1.p2) both map to the alias T1_0. (Ohterwise, you get needless joins
>> like
>> FROM T1 AS T1_0 JOIN T1 AS T1_1 ON T1_0.id=T1_1.id
>> WHERE T1_0.p1=o1 AND T1_1.p1=o2
>>
>> sparqlFrob::object(subject, predicate, object)
>>
>> */
>> %}
>> %name sparqlParser
>> %define LSP_NEEDED
>> %define ERROR_BODY =0
>> %define LEX_BODY =0
>> %header{
>> #include <iostream>
>> #include <string>
>> using namespace std;
>> #undef yyFlexLexer
>> #define yyFlexLexer sparqlFlexLexer
>>
>> #ifndef FLEXFIX
>> #define FLEXFIX YY_sparqlParser_STYPE *val
>> #define FLEXFIX2 val
>> #endif
>>
>> #define MYSQL_YACC
>> #define YYINITDEPTH 100
>> #define YYMAXDEPTH 3200 /* Because of 64K stack */
>> #include "mysql_priv.h"
>> #include "slave.h"
>> #include "lex_symbol.h"
>> #include "item_create.h"
>> #include "sp_head.h"
>> #include "sp_pcontext.h"
>> #include "sp_rcontext.h"
>> #include "sp.h"
>> #include <myisam.h>
>> #include <myisammrg.h>
>>
>> #include "sparqlFrob.h"
>> #define YY_sparqlParser_PARSE_PARAM sparqlFrob *yySparqlFrob
>>
>> #define PLS_REPORT "Please report this bug, along with the query that
>> triggered it."
>>
>> static char * Iso8601formatStr = "%Y-%m-%dT%H:%i:%sZ";
>> //DATE_TIME_FORMAT Iso8601_format = {Iso8601formatStr,
>> strlen(Iso8601formatStr)};
>>
>> extern String Global_string; // call test->print(&Global_string); p
>> Global_string.c_ptr(); call Global_string.free()
>>
>> /* DEBUGGING/DEVELOPMENT
>> * aggregates a string representing the parsed input.
>> */
>> class Buf {
>> protected:
>> char* data;
>> int len;
>> public:
>> Buf() {
>> len = 0;
>> data = 0;
>> }
>> Buf(char *str);
>> Buf(int count, ...);
>> ~Buf() {
>> if (data) delete[] data;
>> }
>> char *str() {return data;}
>> };
>>
>>
>> /* forward class declarations */
>> class BindingContext;
>>
>>
>> /* static function declarations */
>> static const char* scan_string(THD *thd, LEX_STRING *lex_string,
>> const char *start, const char *end, const char look_for);
>>
>>
>> /* Item_POS: overload the mysql Item object for all the SPARQL parts
>> * of speach (URIs, bNodes, variables, and various forms of literal).
>> *
>> * Item overloads: bind_field
>> */
>> class Item_POS
>> {
>> protected:
>> Item *item;
>> public:
>> Item_POS (Item* item_arg) {item = item_arg;}
>> virtual ~Item_POS () {}
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt) =
>> 0;
>> virtual bool is_URI () {return false;}
>> virtual bool is_variable () {return false;}
>> virtual bool is_blank_node () {return false;}
>> virtual bool is_string () {return false;}
>> Item* get_item () {return item;} // OK 'cause ALL Item_POS's are Items.
>> virtual void print(String *str)= 0;
>> };
>>
>>
>> class Item_URI : public Item_field, public Item_POS
>> {
>> protected:
>> LEX_STRING table;
>> LEX_STRING field;
>> LEX_STRING value;
>>
>> public:
>> Item_URI(Name_resolution_context *context_arg, LEX_STRING uri, THD
>> *thd);
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_URI () {return true;}
>> LEX_STRING get_table () {return table;}
>> LEX_STRING get_field () {return field;}
>> LEX_STRING get_value () {return value;}
>> void print(String *str);
>> };
>> extern Item_URI *XSD_integer;
>> extern Item_URI *XSD_float;
>> extern Item_URI *XSD_decimal;
>>
>>
>> /* Item_variable: represent SPARQL variable.
>> *
>> * includes the status of a variable's use in the current JOIN. A variable
>> in
>> * a nested JOIN is different from a variable in an outer context (the
>> * UnionGraphContext::close handles the correlation
>> (U0.foo=oldTable.oldField).
>> *
>> * Item_variable has the same virtual functions as an Item, but is never
>> * added to the SQL compile tree (for instance, in the form of a table
>> * constraint or a select item).
>> */
>> class Item_variable : public Item_field, public Item_POS
>> {
>> private:
>> Name_resolution_context *nr_context;
>> GraphContext *gc;
>> bool bound;
>> bool primary_key;
>> TABLE_LIST *jt;
>> Item_variable *outer;
>> sparqlFrob *frob;
>> const char *real_table_name;
>> int ref_count;
>> const char* get_primary_key () {return
>> frob->get_primary_key(real_table_name);}
>> Item* make_Item_field(Name_resolution_context *context_arg,
>> const char *db_arg, const char *table_name_arg,
>> const char *field_name_arg, const char* real_table_name);
>>
>> public:
>> Item_variable(Name_resolution_context *context_arg, const char
>> *variable_name, Item_variable *outer_arg, sparqlFrob *frob_arg, int
>> ref_count_arg);
>>
>> /* Item_field Virtual Overloads:
>> * Item_variable behaves as as an Item_result_field when the variable
>> * identifies a primary key and as an Item_field the rest of the time.
>> * These methods produce a URI when the constructed from the primary
>> * key field name and value.
>> */
>> void make_field(Send_field *tmp_field);
>> virtual bool fix_fields(THD *, Item **);
>> virtual bool send(Protocol *protocol, String *buffer);
>>
>> /* Item_POS Virtual Overloads:
>> *
>> */
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_variable () {return true;}
>>
>> /* SparqlFrob interface */
>> bool same_variable (const char* variable_name) {
>> return !strcmp(orig_field_name, variable_name);
>> }
>> bool outer_has_no_refs () {return outer ? outer->ref_count == 0 : true;}
>> static Item_variable* match_var_in_list (List<Item_variable>
>> *p_item_variables, const char *var);
>>
>> /* BindingContext interface */
>> void add_ref_count (int ref_count_arg) {ref_count+= ref_count_arg;}
>> Item* get_null_item();
>>
>> /* debugging */
>> void print(String *str);
>> };
>>
>>
>> /* Item_blank_node: represent SPARQL bNode.
>> */
>> class Item_blank_node : public Item_variable
>> {
>> public:
>> Item_blank_node (Name_resolution_context *context_arg, const char
>> *node_name, sparqlFrob *frob_arg) : Item_variable(context_arg,
>> node_name, NULL, frob_arg, 0) {}
>> virtual bool is_blank_node () {return true;}
>> void print(String *str);
>> };
>>
>>
>> /* Item_literal_string: represent SPARQL xsd:string typed literal.
>> */
>> class Item_literal_string : public Item_string, public Item_POS
>> {
>> private:
>> Item_POS *lang_or_datatype;
>> public:
>> Item_literal_string (LEX_STRING string, Item_POS
>> *lang_or_datatype, CHARSET_INFO *charset) : Item_string(string.str,
>> string.length, charset), Item_POS(this) {
>> this->lang_or_datatype = lang_or_datatype;
>> }
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_string () {return true;}
>> Item_POS* get_language_or_datatype () {
>> return lang_or_datatype;
>> }
>> Item_URI* get_datatype () {
>> return lang_or_datatype && lang_or_datatype->is_URI() ?
>> (Item_URI*)lang_or_datatype : NULL;
>> }
>> Item_literal_string* get_language () {
>> return lang_or_datatype && lang_or_datatype->is_string() ?
>> (Item_literal_string*)lang_or_datatype : NULL;
>> }
>> void print(String *str);
>> };
>>
>>
>> /* Item_literal_int: represent SPARQL xsd:integer typed literal.
>> */
>> class Item_literal_int : public Item_int, public Item_POS
>> {
>> public:
>> Item_literal_int (LEX_STRING string, Item_POS *lang_or_datatype,
>> CHARSET_INFO *charset) : Item_int(string.str), Item_POS(this) {}
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_string () {return false;}
>> Item_URI* get_datatype () {return XSD_integer;}
>> Item_literal_string* get_language () {return NULL;}
>> void print(String *str);
>> };
>>
>>
>> /* Item_literal_decimal: represent SPARQL xsd:decimal typed literal.
>> */
>> class Item_literal_decimal : public Item_decimal, public Item_POS
>> {
>> public:
>> Item_literal_decimal (LEX_STRING string, Item_POS *lang_or_datatype,
>> CHARSET_INFO *charset) : Item_decimal(string.str, string.length,
>> charset), Item_POS(this) {}
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_string () {return false;}
>> Item_URI* get_datatype () {return XSD_decimal;}
>> Item_literal_string* get_language () {return NULL;}
>> void print(String *str);
>> };
>>
>>
>> /* Item_literal_float: represent SPARQL xsd:float typed literal.
>> */
>> class Item_literal_float : public Item_float, public Item_POS
>> {
>> public:
>> Item_literal_float (LEX_STRING string, Item_POS *lang_or_datatype,
>> CHARSET_INFO *charset) : Item_float(string.str, string.length),
>> Item_POS(this) {}
>> virtual void bind_field(const char *table_name, const char *alias,
>> const char *field_name_arg, bool trump, THD *thd, TABLE_LIST *jt);
>> virtual bool is_string () {return false;}
>> Item_URI* get_datatype () {return XSD_float;}
>> Item_literal_string* get_language() {return NULL;}
>> void print(String *str);
>> };
>>
>>
>> /* Item_primary_key_field: reference to an Item_field that is a primary
>> key.
>> * Used to create foreign key/primary key constraints on joins.
>> */
>> class Item_primary_key_field : public Item_field
>> {
>> protected:
>> sparqlFrob *frob;
>> const char* real_table_name;
>> public:
>> Item_primary_key_field(Name_resolution_context *context_arg,
>> const char *db_arg, const char *table_name_arg, sparqlFrob
>> *frob_arg, const char* real_table_name_arg);
>> virtual bool fix_fields(THD *, Item **);
>> };
>>
>>
>> #if 0
>> /* Item_func_call: not implemented
>> */
>> class Item_func_call : public Item_POS
>> {
>> protected:
>> Item_POS* name;
>> List<Item>* args;
>> public:
>> Item_func_call (Name_resolution_context *context_arg, Item_POS*
>> name, List<Item>* args) :
>> Item_POS(context_arg, "<Item_func_call>") {
>> this->name = name;
>> this->args = args;
>> }
>> ~Item_func_call() {}
>> void bind_field (const char *table_name, const char *alias, const
>> char *field_name_arg, bool trump, T HD *thd, TABLE_LIST *jt) {}
>> bool fix_fields(THD *, Item **);
>> };
>> #endif
>>
>>
>> /* Alias_info: aliases for a table, indexed by subject.
>> * If it's not in subjects, you need a new alias.
>> */
>> class Alias_info {
>> friend class GraphContext;
>> private:
>> LEX_STRING table;
>> List<Item_POS> subjects;
>> public:
>> Alias_info(LEX_STRING table) {
>> this->table = table;
>> }
>> bool same_table(LEX_STRING table) {
>> return !strcmp(this->table.str, table.str);
>> }
>> };
>>
>>
>> /* BindingContext: SQL table aliases and constraints for a given
>> * SPARQL group graph pattern.
>> */
>> class BindingContext {
>> protected:
>> List<Item_variable> item_variables;
>> // All known table aliases for in the context.
>> List<Alias_info> aliases;
>> Name_resolution_context *name_res_context;
>> BindingContext *parent;
>> LEX *lex;
>> List<Item> *constraints;
>>
>> public:
>> BindingContext(GraphContext *gc_param, BindingContext *parent_param,
>> LEX *lex_param) {
>> parent= parent_param;
>> lex= lex_param;
>> //name_res_context= lex->current_context();
>> }
>> Item_variable* ensure_bound_item_variable(const char *name,
>> sparqlFrob *sparql_frob, int ref_count);
>> void select_star(THD *thd);
>> List<Item_variable>* get_item_variables () { return &item_variables; }
>> Alias_info* get_table_alias(THD *thd, Item_POS *s, Item_URI *p, Item_POS
>> *o);
>> Item* get_bound_variable_constraints(Item *conjunction_constraints,
>> GraphContext *gc);
>> void start_constraints();
>> void end_and_check_constraints() {
>> assert(constraints == lex->select_lex.expr_list.pop());
>> }
>> Item* make_item_constraints() {
>> return constraints->elements ? new Item_cond_and(*constraints) : NULL;
>> }
>> };
>>
>>
>> /* GraphContext: corresponds to a SPARQL graph context.
>> */
>> class GraphContext {
>> protected:
>> GraphContext *parent;
>> THD *thd;
>> Buf *buf; // temporary
>>
>> public:
>> sparqlFrob *sparql_frob; // !!! should be protected
>> GraphContext (GraphContext *parent_param, sparqlFrob *sparql_frob_param)
>> {
>> sparql_frob= sparql_frob_param;
>> parent= parent_param;
>> thd= sparql_frob->thd;
>> buf= NULL;
>> }
>> virtual ~GraphContext () {}
>> virtual void close () {
>> cout << "SPARQL: close {" << endl << buf->str() << "}" << endl;
>> }
>> virtual void no_right_union () {
>> cout << "SPARQL: non-UNION {" << endl << buf->str() << "}" << endl;
>> }
>> void set_Buf (Buf *buf_param) { buf= buf_param; }
>> Buf* get_Buf () { return buf; }
>> virtual bool is_optional () { return false; }
>> virtual BindingContext* get_binding_context () {
>> return parent->get_binding_context();
>> }
>> const char* get_alias_string(Alias_info *ai, Item_POS *s, THD *thd);
>> };
>>
>>
>> /* BindingGraphContext: the graph contexts that require a new
>> BindingContext.
>> * Raison d'etre: factored common code from Union and Graph contexts.
>> */
>> class BindingGraphContext : public GraphContext {
>> friend class Union2GraphContext; // why doesn't BindingGraphContext work
>> here?
>> protected:
>> BindingContext *bindings;
>> SELECT_LEX *current_select;
>> BindingGraphContext(GraphContext *parent_param, sparqlFrob
>> *sparql_frob_param);
>> public:
>> void select_var(Item_variable *var, bool exists);
>> virtual BindingContext* get_binding_context() { return bindings; }
>> };
>>
>>
>> /* RootGraphContext: the graph context just inside the WHERE.
>> */
>> class RootGraphContext : public BindingGraphContext {
>> public:
>> RootGraphContext(GraphContext *parent, sparqlFrob *sparql_frob);
>> virtual void close () {
>> bindings->end_and_check_constraints();
>> }
>> };
>>
>>
>> /* OptionalGraphContext: OPTIONAL { }.
>> * ^^^
>> */
>> class OptionalGraphContext : public GraphContext {
>> public:
>> OptionalGraphContext (GraphContext *parent, sparqlFrob *sparql_frob) :
>> GraphContext(parent, sparql_frob) {
>> /* if (!(on_context= make_join_on_context(thd, $1, $3)))
>> assert(0);
>> thd->lex->push_context(on_context);
>> */
>> }
>> virtual void close () {
>> /* thd->lex->pop_context();
>> */
>> }
>> virtual bool is_optional () { return true; }
>> };
>>
>>
>> /* Union1GraphContext: { } UNION { }.
>> * ^^^
>> */
>> class Union1GraphContext : public BindingGraphContext {
>> public:
>> Union1GraphContext(GraphContext *parent, sparqlFrob *sparql_frob);
>> virtual void no_right_union () {
>> /* Just a spurious {} so copy the context up to the parent.
>> @@@ TODO
>> */
>> my_printf_error(1, "SPARQL: implementation limitation: not ready to
>> deal with non-UNION %s. "PLS_REPORT, MYF(0), buf->str());
>> }
>> virtual void close();
>> Item* map_variables_to_outer_context () { // !!! does nothing
>> return NULL;
>> }
>> };
>>
>>
>> /* Union1GraphContext: { } UNION { }.
>> * ^^^
>> */
>> class Union2GraphContext : public BindingGraphContext {
>> private:
>> BindingGraphContext *left; // left side of the UNION
>>
>> public:
>> Union2GraphContext(GraphContext *parent, sparqlFrob *sparql_frob);
>> virtual void close();
>> };
>>
>>
>> /* GraphGraphContext: GRAPH { }.
>> * ^^^
>> */
>> class GraphGraphContext : public BindingGraphContext {
>> public:
>> GraphGraphContext (GraphContext *parent, sparqlFrob *sparql_frob) :
>> BindingGraphContext(parent, sparql_frob) {}
>> };
>>
>>
>> %}
>>
>> %union {
>> int num;
>> LEX_STRING lex_str;
>> Item *item;
>> List<Item> *item_list;
>> List<BindingGraphContext> *BindingGraphContext_list;
>> Item_POS *item_pos;
>> GraphContext *graph_context;
>> BindingGraphContext *binding_graph_context;
>> Buf *buf;
>> }
>>
>> %token <buf> IT_BASE
>> %token <buf> IT_PREFIX
>> %token <buf> IT_SELECT
>> %token <buf> IT_DISTINCT
>> %token <buf> GT_TIMES
>> %token <buf> IT_CONSTRUCT
>> %token <buf> IT_DESCRIBE
>> %token <buf> IT_ASK
>> %token <buf> IT_FROM
>> %token <buf> IT_NAMED
>> %token <buf> IT_WHERE
>> %token <buf> IT_ORDER
>> %token <buf> IT_BY
>> %token <buf> IT_ASC
>> %token <buf> IT_DESC
>> %token <buf> IT_LIMIT
>> %token <buf> IT_OFFSET
>> %token <buf> GT_LCURLEY
>> %token <buf> GT_RCURLEY
>> %token <buf> GT_DOT
>> %token <buf> IT_OPTIONAL
>> %token <buf> IT_GRAPH
>> %token <buf> IT_UNION
>> %token <buf> IT_FILTER
>> %token <buf> GT_COMMA
>> %token <buf> GT_LPAREN
>> %token <buf> GT_RPAREN
>> %token <buf> GT_SEMI
>> %token <item_pos> IT_a
>> %token <buf> GT_LBRACKET
>> %token <buf> GT_RBRACKET
>> %token <buf> GT_MINUS
>> %token <buf> GT_PLUS
>> %token <buf> GT_OR
>> %token <buf> GT_AND
>> %token <buf> GT_EQUAL
>> %token <buf> GT_NEQUAL
>> %token <buf> GT_LT
>> %token <buf> GT_GT
>> %token <buf> GT_LE
>> %token <buf> GT_GE
>> %token <buf> GT_DIVIDE
>> %token <buf> GT_NOT
>> %token <buf> IT_STR
>> %token <buf> IT_LANG
>> %token <buf> IT_LANGMATCHES
>> %token <buf> IT_DATATYPE
>> %token <buf> IT_BOUND
>> %token <buf> IT_isIRI
>> %token <buf> IT_isURI
>> %token <buf> IT_isBLANK
>> %token <buf> IT_isLITERAL
>> %token <buf> IT_REGEX
>> %token <buf> GT_DTYPE
>> %token <item_pos> IT_true
>> %token <item_pos> IT_false
>> %token <lex_str> Q_IRI_REF
>> %token <lex_str> QNAME_NS
>> %token <lex_str> QNAME
>> %token <lex_str> BLANK_NODE_LABEL
>> %token <lex_str> VAR1
>> %token <lex_str> VAR2
>> %token <item_pos> LANGTAG
>> %token <lex_str> INTEGER
>> %token <lex_str> DECIMAL
>> %token <lex_str> DOUBLE
>> %token <lex_str> STRING_LITERAL1
>> %token <lex_str> STRING_LITERAL2
>> %token <lex_str> STRING_LITERAL_LONG1
>> %token <lex_str> STRING_LITERAL_LONG2
>> %token <item_pos> NIL
>> %token <lex_str> ANON
>> %type <buf> Query
>> %type <buf>
>> _O_QSelectQuery_E__Or__QConstructQuery_E__Or__QDescribeQuery_E__Or__QAskQuery_E__C
>> %type <buf> Prolog
>> %type <buf> _QBaseDecl_E_Opt
>> %type <buf> _QPrefixDecl_E_Star
>> %type <buf> BaseDecl
>> %type <buf> PrefixDecl
>> %type <buf> SelectQuery
>> %type <buf> _QDISTINCT_E_Opt
>> %type <buf> _QVar_E_Plus
>> %type <buf> _O_QVar_E_Plus_Or__QTIMES_E__C
>> %type <buf> _QDatasetClause_E_Star
>> %type <buf> ConstructQuery
>> %type <buf> DescribeQuery
>> %type <buf> _QVarOrIRIref_E_Plus
>> %type <buf> _O_QVarOrIRIref_E_Plus_Or__QTIMES_E__C
>> %type <buf> _QWhereClause_E_Opt
>> %type <buf> AskQuery
>> %type <buf> DatasetClause
>> %type <buf> _O_QDefaultGraphClause_E__Or__QNamedGraphClause_E__C
>> %type <buf> DefaultGraphClause
>> %type <buf> NamedGraphClause
>> %type <buf> SourceSelector
>> %type <buf> WhereClause
>> %type <buf> _QWHERE_E_Opt
>> %type <buf> SolutionModifier
>> %type <buf> _QOrderClause_E_Opt
>> %type <buf> _QLimitClause_E_Opt
>> %type <buf> _QOffsetClause_E_Opt
>> %type <buf> OrderClause
>> %type <buf> _QOrderCondition_E_Plus
>> %type <buf> OrderCondition
>> %type <num> _O_QASC_E__Or__QDESC_E__C
>> %type <buf> _O_QASC_E__Or__QDESC_E____QBrackettedExpression_E__C
>> %type <item>
>> _O_QFunctionCall_E__Or__QVar_E__Or__QBrackettedExpression_E__C
>> %type <buf> LimitClause
>> %type <buf> OffsetClause
>> %type <graph_context> GroupGraphPattern
>> %type <buf> GraphPattern
>> %type <buf> _QDOT_E_Opt
>> %type <buf>
>> _O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C
>> %type <buf>
>> _Q_O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C_E_Opt
>> %type <buf> FilteredBasicGraphPattern
>> %type <buf> _QBlockOfTriples_E_Opt
>> %type <buf>
>> _O_QConstraint_E____QDOT_E_Opt___QFilteredBasicGraphPattern_E__C
>> %type <buf>
>> _Q_O_QConstraint_E____QDOT_E_Opt___QFilteredBasicGraphPattern_E__C_E_Opt
>> %type <buf> BlockOfTriples
>> %type <buf> _QTriplesSameSubject_E_Opt
>> %type <buf> _O_QDOT_E____QTriplesSameSubject_E_Opt_C
>> %type <buf> _Q_O_QDOT_E____QTriplesSameSubject_E_Opt_C_E_Star
>> %type <buf> GraphPatternNotTriples
>> %type <buf> OptionalGraphPattern
>> %type <buf> GraphGraphPattern
>> %type <buf> GroupOrUnionGraphPattern
>> %type <binding_graph_context> _O_QUNION_E____QGroupGraphPattern_E__C
>> %type <BindingGraphContext_list>
>> _Q_O_QUNION_E____QGroupGraphPattern_E__C_E_Star
>> %type <buf> Constraint
>> %type <item>
>> _O_QBrackettedExpression_E__Or__QBuiltInCall_E__Or__QFunctionCall_E__C
>> %type <item> FunctionCall
>> %type <item_list> ArgList
>> %type <item> _O_QCOMMA_E____QExpression_E__C
>> %type <NONE> _Q_O_QCOMMA_E____QExpression_E__C_E_Star
>> %type <item_list>
>> _O_QNIL_E__Or__QLPAREN_E____QExpression_E____QCOMMA_E____QExpression_E_Star___QRPAREN_E__C
>> %type <buf> ConstructTemplate
>> %type <buf> ConstructTriples
>> %type <buf> _O_QDOT_E____QConstructTriples_E__C
>> %type <buf> _Q_O_QDOT_E____QConstructTriples_E__C_E_Opt
>> %type <buf>
>> _O_QTriplesSameSubject_E____QDOT_E____QConstructTriples_E_Opt_C
>> %type <buf>
>> _Q_O_QTriplesSameSubject_E____QDOT_E____QConstructTriples_E_Opt_C_E_Opt
>> %type <buf> TriplesSameSubject
>> %type <buf> PropertyList
>> %type <buf> _QPropertyListNotEmpty_E_Opt
>> %type <buf> PropertyListNotEmpty
>> %type <buf> _O_QSEMI_E____QPropertyList_E__C
>> %type <buf> _Q_O_QSEMI_E____QPropertyList_E__C_E_Opt
>> %type <buf> ObjectList
>> %type <buf> _O_QCOMMA_E____QObjectList_E__C
>> %type <buf> _Q_O_QCOMMA_E____QObjectList_E__C_E_Opt
>> %type <item_pos> Verb
>> %type <item_pos> TriplesNode
>> %type <item_pos> BlankNodePropertyList
>> %type <item_pos> Collection
>> %type <buf> _QGraphNode_E_Plus
>> %type <item_pos> GraphNode
>> %type <item_pos> VarOrTerm
>> %type <item_pos> VarOrIRIref
>> %type <buf> VarOrBlankNodeOrIRIref
>> %type <item_pos> Var
>> %type <item_pos> GraphTerm
>> %type <buf> _O_QMINUS_E__Or__QPLUS_E__C
>> %type <buf> _Q_O_QMINUS_E__Or__QPLUS_E__C_E_Opt
>> %type <item> Expression
>> %type <item> ConditionalOrExpression
>> %type <NONE> _O_QOR_E____QConditionalAndExpression_E__C
>> %type <NONE> _Q_O_QOR_E____QConditionalAndExpression_E__C_E_Star
>> %type <item> ConditionalAndExpression
>> %type <NONE> _O_QAND_E____QValueLogical_E__C
>> %type <NONE> _Q_O_QAND_E____QValueLogical_E__C_E_Star
>> %type <item> ValueLogical
>> %type <item> RelationalExpression
>> %type <NONE>
>> _O_QEQUAL_E____QNumericExpression_E__Or__QNEQUAL_E____QNumericExpression_E__Or__QLT_E____QNumericExpression_E__Or__QGT_E____QNumericExpression_E__Or__QLE_E____QNumericExpression_E__Or__QGE_E____QNumericExpression_E__C
>> %type <NONE>
>> _Q_O_QEQUAL_E____QNumericExpression_E__Or__QNEQUAL_E____QNumericExpression_E__Or__QLT_E____QNumericExpression_E__Or__QGT_E____QNumericExpression_E__Or__QLE_E____QNumericExpression_E__Or__QGE_E____QNumericExpression_E__C_E_Opt
>> %type <item> NumericExpression
>> %type <item> AdditiveExpression
>> %type <NONE>
>> _O_QPLUS_E____QMultiplicativeExpression_E__Or__QMINUS_E____QMultiplicativeExpression_E__C
>> %type <NONE>
>> _Q_O_QPLUS_E____QMultiplicativeExpression_E__Or__QMINUS_E____QMultiplicativeExpression_E__C_E_Star
>> %type <item> MultiplicativeExpression
>> %type <NONE>
>> _O_QTIMES_E____QUnaryExpression_E__Or__QDIVIDE_E____QUnaryExpression_E__C
>> %type <NONE>
>> _Q_O_QTIMES_E____QUnaryExpression_E__Or__QDIVIDE_E____QUnaryExpression_E__C_E_Star
>> %type <item> UnaryExpression
>> %type <item> PrimaryExpression
>> %type <item> BrackettedExpression
>> %type <item> BuiltInCall
>> %type <item> RegexExpression
>> %type <item> _Q_O_QCOMMA_E____QExpression_E__C_E_Opt
>> %type <item> IRIrefOrFunction
>> %type <item_list> _QArgList_E_Opt
>> %type <item_pos> RDFLiteral
>> %type <item_pos> _O_QDTYPE_E____QIRIref_E__C
>> %type <item_pos> _O_QLANGTAG_E__Or__QDTYPE_E____QIRIref_E__C
>> %type <item_pos> _Q_O_QLANGTAG_E__Or__QDTYPE_E____QIRIref_E__C_E_Opt
>> %type <item_pos> NumericLiteral
>> %type <item_pos> BooleanLiteral
>> %type <lex_str> String
>> %type <item_pos> IRIref
>> %type <item_pos> QName
>> %type <item_pos> BlankNode
>> %%
>>
>> Query:
>> {
>> LEX *lex= yySparqlFrob->thd->lex;
>> SELECT_LEX *sel= lex->current_select;
>> lex->current_select->parsing_place = NO_MATTER;
>> sel->offset_limit= NULL;
>> sel->select_limit= NULL;
>> // create root context right away so SELECTs have a context
>> yySparqlFrob->push_context(
>> new RootGraphContext(yySparqlFrob->head_context(), yySparqlFrob));
>> }
>> Prolog
>> _O_QSelectQuery_E__Or__QConstructQuery_E__Or__QDescribeQuery_E__Or__QAskQuery_E__C
>> {
>> Buf *b = new Buf(2, $2, $3);
>> cout << "Query end -- " << b->str() << endl;
>> delete b;
>> };
>>
>> _O_QSelectQuery_E__Or__QConstructQuery_E__Or__QDescribeQuery_E__Or__QAskQuery_E__C:
>> SelectQuery {$$ = $1}
>> | ConstructQuery {$$ = $1; YYABORT;}
>> | DescribeQuery {$$ = $1; YYABORT;}
>> | AskQuery {$$ = $1; YYABORT;};
>>
>> Prolog:
>> _QBaseDecl_E_Opt _QPrefixDecl_E_Star {$$ = new Buf(2, $1, $2)};
>>
>> _QBaseDecl_E_Opt:
>> {$$ = new Buf()}
>> | BaseDecl {$$ = $1};
>>
>> _QPrefixDecl_E_Star:
>> {$$ = new Buf()}
>> | _QPrefixDecl_E_Star PrefixDecl {$$ = new Buf(2, $1, $2)};
>>
>> BaseDecl:
>> IT_BASE Q_IRI_REF {$$ = new Buf(2, new Buf("BASE"), new Buf($2.str))};
>>
>> PrefixDecl:
>> IT_PREFIX QNAME_NS Q_IRI_REF {$$ = new Buf(3, $1, new Buf($2.str), new
>> Buf($3.str))};
>>
>> SelectQuery:
>> IT_SELECT
>> {
>> LEX *lex= yySparqlFrob->lex;
>> SELECT_LEX *sel= lex->current_select;
>>
>> sel->parsing_place = SELECT_LIST;
>> lex->sql_command= SQLCOM_SELECT; // SQLCOM_EMPTY_QUERY;
>> }
>> _QDISTINCT_E_Opt _O_QVar_E_Plus_Or__QTIMES_E__C
>> {
>> //SELECT_LEX *sel= yySparqlFrob->lex->current_select;
>>
>> // commented out 'cause the didn't seem to be needed - EGP 20060326
>> // lex->create_view_select_start = $1->get_item()->name;
>> //sel->parsing_place = NO_MATTER;
>> //sel->set_braces(0);
>> }
>> _QDatasetClause_E_Star WhereClause SolutionModifier
>> {
>> LEX *lex= yySparqlFrob->lex;
>> SELECT_LEX *sel= lex->current_select;
>>
>> // Now that we have all the variables, fix up the SELECTs.
>> yySparqlFrob->select_variables();
>>
>> sel->parsing_place = NO_MATTER;
>> $$ = new Buf(6, new Buf("SELECT"), $3, $4, $6, $7, $8);
>> }
>> ;
>>
>> _QDISTINCT_E_Opt:
>> {$$ = new Buf()}
>> | IT_DISTINCT
>> {
>> yySparqlFrob->set_distinct(true);
>> $$ = $1;
>> };
>>
>> _QVar_E_Plus:
>> Var
>> {
>> THD *thd= yySparqlFrob->thd;
>> LEX *lex= thd->lex;
>> SELECT_LEX *sel= lex->current_select;
>> //sel->use_index_ptr=sel->ignore_index_ptr=0;
>> sel->table_join_options= 0;
>> if (add_item_to_list(thd, $1->get_item())) {
>> YYABORT;
>> }
>>
>> $$ = new Buf($1->get_item()->name);
>> }
>> | _QVar_E_Plus Var
>> {
>> THD *thd= yySparqlFrob->thd;
>> if (add_item_to_list(thd, $2->get_item())) {
>> YYABORT;
>> }
>> $$ = new Buf(2, $1, new Buf($2->get_item()->name));
>> };
>>
>> _O_QVar_E_Plus_Or__QTIMES_E__C:
>> _QVar_E_Plus {$$ = $1}
>> | GT_TIMES
>> { $$ = $1; }
>> ;
>>
>> _QDatasetClause_E_Star:
>> {$$ = new Buf()}
>> | _QDatasetClause_E_Star DatasetClause {$$ = new Buf(2, $1, $2)};
>>
>> ConstructQuery:
>> IT_CONSTRUCT ConstructTemplate _QDatasetClause_E_Star WhereClause
>> SolutionModifier {$$ = new Buf(5, $1, $2, $3, $4, $5)};
>>
>> DescribeQuery:
>> IT_DESCRIBE _O_QVarOrIRIref_E_Plus_Or__QTIMES_E__C
>> _QDatasetClause_E_Star _QWhereClause_E_Opt SolutionModifier {$$ = new
>> Buf(5, $1, $2, $3, $4, $5)};
>>
>> _QVarOrIRIref_E_Plus:
>> VarOrIRIref {$$ = new Buf($1->get_item()->name)}
>> | _QVarOrIRIref_E_Plus VarOrIRIref {$$ = new Buf(2, $1, new
>> Buf($2->get_item()->name))};
>>
>> _O_QVarOrIRIref_E_Plus_Or__QTIMES_E__C:
>> _QVarOrIRIref_E_Plus {$$ = $1}
>> | GT_TIMES {$$ = $1};
>>
>> _QWhereClause_E_Opt:
>> {$$ = new Buf()}
>> | WhereClause {$$ = $1};
>>
>> AskQuery:
>> IT_ASK _QDatasetClause_E_Star WhereClause {$$ = new Buf(3, $1, $2,
>> $3)};
>>
>> DatasetClause:
>> IT_FROM _O_QDefaultGraphClause_E__Or__QNamedGraphClause_E__C {$$ = new
>> Buf(2, $1, $2)};
>>
>> _O_QDefaultGraphClause_E__Or__QNamedGraphClause_E__C:
>> DefaultGraphClause {$$ = $1}
>> | NamedGraphClause {$$ = $1};
>>
>> DefaultGraphClause:
>> SourceSelector {$$ = $1};
>>
>> NamedGraphClause:
>> IT_NAMED SourceSelector {$$ = new Buf(2, $1, $2)};
>>
>> SourceSelector:
>> IRIref {$$ = new Buf($1->get_item()->name)};
>>
>> WhereClause:
>> _QWHERE_E_Opt
>> {
>> SELECT_LEX* sel = &yySparqlFrob->lex->select_lex;
>> sel->parsing_place = IN_WHERE;
>>
>> /* we've been using a context (for SELECTs) but need
>> to pop it so it can be pushed back on at the LCURLEY */
>> yySparqlFrob->set_next_GC(yySparqlFrob->pop_context());
>> }
>> GroupGraphPattern
>> {
>> GraphContext *gc= $3;
>> SELECT_LEX* sel = &yySparqlFrob->lex->select_lex;
>> BindingContext *bc= gc->get_binding_context();
>> Item *constraints= bc->make_item_constraints();
>> yySparqlFrob->add_root_constraints(bc->
>> get_bound_variable_constraints(constraints, gc));
>> Item *where = yySparqlFrob->get_root_constraints();
>> if (where) {
>> where->top_level_item();
>> sel->where = where;
>> String string;
>> where->print(&string);
>> $$ = new Buf(4, $1, $3->get_Buf(), new Buf("WHERE"), new
>> Buf(string.c_ptr()));
>> } else {
>> $$ = new Buf(2, $1, $3->get_Buf());
>> }
>> sel->parsing_place= NO_MATTER;
>> };
>>
>> _QWHERE_E_Opt:
>> {$$ = new Buf()}
>> | IT_WHERE {$$ = $1};
>>
>> SolutionModifier:
>> _QOrderClause_E_Opt _QLimitClause_E_Opt _QOffsetClause_E_Opt {$$ = new
>> Buf(3, $1, $2, $3)};
>>
>> _QOrderClause_E_Opt:
>> {$$ = new Buf()}
>> | OrderClause {$$ = $1};
>>
>> _QLimitClause_E_Opt:
>> {$$ = new Buf();}
>> | LimitClause {$$ = $1};
>>
>> _QOffsetClause_E_Opt:
>> {$$ = new Buf()}
>> | OffsetClause {$$ = $1};
>>
>> OrderClause:
>> IT_ORDER IT_BY _QOrderCondition_E_Plus {$$ = new Buf(3, $1, $2, $3)};
>>
>> _QOrderCondition_E_Plus:
>> OrderCondition {$$ = $1}
>> | _QOrderCondition_E_Plus OrderCondition {$$ = new Buf(2, $1, $2)};
>>
>> OrderCondition:
>> _O_QASC_E__Or__QDESC_E____QBrackettedExpression_E__C {$$ = $1}
>> | _O_QFunctionCall_E__Or__QVar_E__Or__QBrackettedExpression_E__C {$$ =
>> new Buf("!!!")};
>>
>> _O_QASC_E__Or__QDESC_E__C:
>> IT_ASC {$$ =1}
>> | IT_DESC {$$ =0};
>>
>> _O_QASC_E__Or__QDESC_E____QBrackettedExpression_E__C:
>> _O_QASC_E__Or__QDESC_E__C BrackettedExpression
>> {
>> if (add_order_to_list(yySparqlFrob->thd, $2, (bool) $1))
>> YYABORT; String string;
>> $2->print(&string);
>> $$ = new Buf(2, new Buf((char*)($1 ? "ASK" : "DESC")), new
>> Buf(string.c_ptr()));
>> };
>>
>> _O_QFunctionCall_E__Or__QVar_E__Or__QBrackettedExpression_E__C:
>> FunctionCall {$$ = $1}
>> | Var {$$ = $1->get_item()}
>> | BrackettedExpression {$$ = $1};
>>
>> LimitClause:
>> IT_LIMIT INTEGER
>> {
>> SELECT_LEX *sel = &yySparqlFrob->lex->select_lex;
>> sel->select_limit = new Item_uint($2.str, $2.length);
>> sel->explicit_limit= 1;
>> $$ = new Buf(2, $1, new Buf($2.str));
>> };
>>
>> OffsetClause:
>> IT_OFFSET INTEGER
>> {
>> SELECT_LEX *sel = &yySparqlFrob->lex->select_lex;
>> sel->offset_limit = new Item_uint($2.str, $2.length);
>> sel->explicit_limit= 1;
>> $$ = new Buf(2, $1, new Buf($2.str));
>> };
>>
>> GroupGraphPattern:
>> GT_LCURLEY
>> {
>> GraphContext *gc = yySparqlFrob->was_next_GC();
>> yySparqlFrob->push_context(gc ? gc : new
>> Union1GraphContext(yySparqlFrob->head_context(), yySparqlFrob));
>> }
>> GraphPattern GT_RCURLEY
>> {
>> if (yySparqlFrob->get_distinct())
>> yySparqlFrob->lex->current_select->options|= SELECT_DISTINCT;
>> $$= yySparqlFrob->pop_context();
>> $$->set_Buf(new Buf(3, $1, $3, $4));
>> $$->close();
>> };
>>
>> GraphPattern:
>> FilteredBasicGraphPattern
>> _Q_O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C_E_Opt {$$
>> = new Buf(2, $1, $2)};
>>
>> _QDOT_E_Opt:
>> {$$ = new Buf()}
>> | GT_DOT {$$ = $1};
>>
>> _O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C:
>> GraphPatternNotTriples _QDOT_E_Opt GraphPattern {$$ = new Buf(3, $1,
>> $2, $3)};
>>
>> _Q_O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C_E_Opt:
>> {$$ = new Buf()}
>> | _O_QGraphPatternNotTriples_E____QDOT_E_Opt___QGraphPattern_E__C {$$
>> = $1};
>>
>> FilteredBa