Sphinx Search C# .NET Client API
Need a C# native client API for Sphinx Search to use in your C# or VB ASP.NET projects? So did I, so I wrote one.
Yesterday, I found Sphinx Search and decided to try implementing it in place of the (horrid) MySql Fulltext searching for my Photocore project. After downloading the binaries and indexing a couple million rows of metadata, I was amazed at how well it performs. It indexed all my data in less than a minute (compared to the 30 minutes required by MySql Fulltext) and I haven’t come up with a search that takes longer than 0.005 seconds. I was hooked immediately. So I needed a .NET API because I didn’t want to patch my database server to use the Sphinx plugin.
Source download after the jump.
Download v0.2 source | binaries
Download has no warranties, blah, blah, blah. It’s based on the Java API and is released under the GNU GPL license.
Important Note
It does not have all the features implemented. Most notably, it does not have support for Exceprts. If you modify code to make it work, send it back and I’ll post the updated version here. Or, if I get enough requests to implement it, I might. But my use of Sphinx does not include excerpts, so I didn’t bother at this point.
Other Notes
This is a pretty quick-and-dirty project, I’ll proabably clean it up over time and repost new versions, but for now make sure you test it thoroughly before putting it into production on your own sites. That’s your responsibility…
It is tested against the Sphinx searchd server version 0.9.8-dev running on both linux (ubuntu) and windows (vista) with simple queries only. I didn’t do anything fancy in my queries.
It does not work against 0.9.7 or earlier because those did not support multiple queries. It should work against future versions of Sphinx server. No promises though.
Usage
Super-simple to use after you have a searchd server running with an index. Three lines will do a search for “rose bowl gooley” matching any index (“*”) and will get the second page of results (items 41-80) with a max result count of 10,000 documents.
// set hostname and port defaults
SphinxClient cli = new SphinxClient(hostname, 3312);
// set the pagination params
// (this is page 2, 40 per page, 10000 max)
cli.SetLimits(40, 40, 10000);
// define the query and index
// ("*" means any index)
cli.AddQuery("rose bowl gooley", "*");
// run the query and get the results
SphinxResult[] results = cli.RunQueries();
The SphinxResult object contains an array of SphinxMatches which will provide you with the DocumentId numbers for all matches. Simple as that.
Happy searching!
October 13th, 2009 at 1:55 am
Check this client too http://code.google.com/p/sphinx-dotnet-client/
It’s open source and supports all latest features and Sphinx server protocols,
ADO.NET like API. Provided with samples and Sphinx GUI index test tool.
October 5th, 2009 at 11:07 am
Hey everyone, I re-wrote a .NET client for Sphinx from scratch this past weekend and have posted it up on Google Code (http://code.google.com/p/sphinxdotnet) Feel free to join the project and help out if you want. I think it is feature complete for the sphinxsearch trunk (release 2011)
August 28th, 2009 at 2:27 pm
One more piece:
public void SetFilterFloatRange(string attribute, float min, float max, bool exclude)
{
Debug.Assert ( min<=max, “min must be less or equal to max” );
try
{
BinaryWriter bw = new BinaryWriter(_filterStreamData);
WriteToStream(bw, attribute);
WriteToStream(bw, SPH_FILTER_FLOATRANGE);
WriteToStream(bw, min);
WriteToStream(bw, max);
WriteToStream(bw, exclude ? 1 : 0);
}
catch ( Exception e )
{
Debug.Assert ( false, “IOException: ” + e.ToString() );
}
_filterCount++;
}
August 28th, 2009 at 2:01 pm
Also added UpdateAttributes API Call:
/**
* Connect to searchd server and update given attributes on given documents in given indexes.
* Sample code that will set group_id=123 where id=1 and group_id=456 where id=3:
*
*
* String[] attrs = new String[1];
*
* attrs[0] = “group_id”;
* long[][] values = new long[2][2];
*
* values[0] = new long[2]; values[0][0] = 1; values[0][1] = 123;
* values[1] = new long[2]; values[1][0] = 3; values[1][1] = 456;
*
* int res = cl.UpdateAttributes ( “test1″, attrs, values );
*
*
* @param index index name(s) to update; might be distributed
* @param attrs array with the names of the attributes to update
* @param values array of updates; each long[] entry must contains document ID
* in the first element, and all new attribute values in the following ones
* @return -1 on failure, amount of actually found and updated documents (might be 0) on success
*
* @throws SphinxException on invalid parameters
*/
public int UpdateAttributes ( String index, String[] attrs, long[][] values )
{
/* check args */
Debug.Assert ( !string.IsNullOrEmpty(index), “no index name provided” );
Debug.Assert ( attrs!=null && attrs.Length>0, “no attribute names provided” );
Debug.Assert ( values!=null && values.Length>0, “no update entries provided” );
for ( int i=0; i<values.Length; i++ )
{
Debug.Assert ( values[i]!=null, “update entry #” + i + ” is null” );
Debug.Assert ( values[i].Length == 1 + attrs.Length, “update entry #” + i + ” has wrong length” );
}
/* build and send request */
if (_client == null || !_client.Connected)
{
this.Connect();
}
if (_client == null) return -1;
StreamReader sr = new StreamReader(_client.GetStream());
BinaryWriter sw = new BinaryWriter(_client.GetStream());
BinaryWriter tsw = new BinaryWriter(new MemoryStream()); /* temp command buffer */
try
{
WriteToStream(sw, VER_MAJOR_PROTO);
WriteToStream(sw, (short)SEARCHD_COMMAND_UPDATE);
WriteToStream(sw, (short)VER_COMMAND_UPDATE);
this.WriteToStream(tsw, index);
this.WriteToStream(tsw, attrs.Length);
for (int i = 0; i< attrs.Length; i++)
{
this.WriteToStream(tsw, attrs[i]);
}
this.WriteToStream(tsw, values.Length);
for (int i = 0; i < values.Length; i++)
{
this.WriteToStream(tsw, values[i][0]); /* send docid as 64bit value */
for (int j = 1; j < values[i].Length; j++)
{
this.WriteToStream(tsw, (int)values[i][j]); /* send values as 32bit values; FIXME! what happens when they are over 2^31? */
}
}
this.WriteToStream(sw, (int)tsw.BaseStream.Length); // send the request size
sw.Write(((MemoryStream)tsw.BaseStream).ToArray());
sw.Flush();
} catch ( Exception e )
{
_error = “internal error: failed to build request: ” + e;
return -1;
}
/* get and parse response */
byte[] response = GetResponse(_client, VER_COMMAND_UPDATE);
if ( response == null )
return -1;
try
{
BinaryReader br = new BinaryReader(new MemoryStream(response));
return this.ReadInt32(br);
} catch ( Exception )
{
_error = “incomplete reply”;
return -1;
}
}
August 28th, 2009 at 1:53 pm
Added support for floats. Here’s some code:
private void WriteToStream(BinaryWriter sw, float data)
{
int intBits = FloatToInt32Bits(data);
byte[] d = BitConverter.GetBytes(intBits);
sw.Write(_Reverse(d));
}
public static unsafe int FloatToInt32Bits(float value)
{
return *(((int*)&value));
}
public static unsafe float Int32FloatToBits(int value)
{
return *(((float*)&value));
}
July 24th, 2009 at 3:02 am
Great initiative Christoper. Any chance you’ll be putting up a shared project somewhere? Google code? Github? I’m sure others are willing to contribute as well.
July 8th, 2009 at 10:23 pm
We are looking for a solution for Sphinx to run on 64bits Windows Server. Any clues? Don’t even mind a fee for a compilation that will work on 64bits windows.
June 30th, 2009 at 9:38 pm
Hello,sir
There is a problem with getting a match’s attrValues.its always be nothing.
How can I init a match’s attrValues array.
June 24th, 2009 at 9:40 am
Hi,
I had change the port according to the server but getting error
“Connect: Read from socket failed: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.”
Plz help…
and also can you please let me know how we can assure that whichever port i m using is the correct one?.
Thanks
Mohit
June 5th, 2009 at 1:16 am
Great to see this project. Even better would be a GitHub project and better contributions. “I fixed some things” doesn’t help other users, unfortunately.
May 12th, 2009 at 7:44 am
For unsigned integer attributes to appear correctly you need to change:
/* handle everything else as unsigned ints */
long val = bw.ReadUInt32();
to
/* handle everything else as unsigned ints */
uint val = ReadUInt32(bw);
to make it go through byte reversal. (no need to ask me for the source!). Also easy to add the latest enumerations (and change the validation code) to use options in 0.9.9.
December 8th, 2008 at 4:18 pm
Hi,
I noticed that every time I run a query, a new Tcp connection is established. Opening a new connection lasts 1000~ milliseconds.
It makes 1 second delay for querying results, and there I lost the super speed of Sphinx.
What do you recommend in order to improve performance ?
Thanks,
Itye.
December 4th, 2008 at 12:23 pm
[quote comment="3017"]Hello sir,
there is a small problem with the attributes reading – the numbers are not real. I’ve fixed it. If anyone need these sources, I’m here.[/quote]
I am also interested in this fix: peter at emblemsoftware dot com
Thanks.
October 4th, 2008 at 12:05 pm
[quote comment="3017"]Hello sir,
there is a small problem with the attributes reading – the numbers are not real. I’ve fixed it. If anyone need these sources, I’m here.[/quote]
Can you send it to miha at studiopesec dot com.
Tnx in advance
Miha
June 30th, 2009 at 9:42 pm
Maybe my problem is the same to you.
When can the version be published?
September 11th, 2008 at 4:24 am
hi ankesh -
that sounds like a firewall blocking your traffic or the wrong port you’re trying to use. make sure your sphinx server is at the hostname you pass to the SphinxClient constructor, and make sure the server is using the default port 3312 or change the port to match your server.
that TCP error probably has nothing to do with the client API code.
hope that helps.
September 11th, 2008 at 1:15 am
Hi
I am trying your code but not able to connect through TCP .The message are
“No connection could be made because the target machine actively refused it”
Please help me ASAP.
August 27th, 2008 at 1:31 pm
is SphinxClient thread safe? Can I make a static SphinxClient and use it across threads?
March 25th, 2008 at 5:09 am
Hello sir,
there is a small problem with the attributes reading – the numbers are not real. I’ve fixed it. If anyone need these sources, I’m here.
January 17th, 2008 at 12:21 pm
hey Charli -
Glad to hear that it might be useful to you. I’m going production with it this weekend and it has done well in all my tests thus far.
Let me know how it works for you (and where you’re using it, since I’m curious).
cheers!
January 16th, 2008 at 12:30 pm
I just stumbled into your website while checking out sphinx on Google, thought someone might have implemented sphinx using a .NET language, and here it is. This is really good news, I haven’t tried your code yet but by the quality of it I’m guessing it’s working great.
We haven’t (until now) been able to implement full text search on our forum since mysql fulltext search is way to slow, and I haven’t been a fan of patching mysql with sphinx since software I patch usually tend to work real bad. I’m gonna play around with your code together with sphinx to see how it works :).