layer:11

an addiction to or an obsession with acquiring, manipulating, and sharing information

Connecting to Cassandra with C# and Thrift

Here’s something I wanted to write down for a long time now, because every time I have to do this (when Cassandra changes its interface), I have already forgotten what and exactly how needs to be done. Let’s assume that we have Cassandra up and running, because most distributions should have a package ready to use, or at least a build script. And now we want to connect to it with C#. What we need is two C# libraries, Thrift and Cassandra. So, let’s build them from source.

What exactly is Thrift? It’s a framework for building services and clients for those services - you write a file that describes what you want, then use the Thrift compiler to get a server skeleton and a RPC client in any language that Thrift supports.

First thing we need is a Thrift compiler and a Thrift library for C#. Let’s download the Thrift source and simply ./configure && make it, no need to install. You might want to disable support for other languages, they don’t get in the way, but I had problems building a Ruby library from Thrift 0.7.0. Now we have a C# Thrift library (lib/csharp/Thrift.dll) and a Thrift compiler (compiler/cpp/thrift). This library doesn’t give us much by itself, but it’s required for the Cassandra library, that we’ll generate with the Thrift compiler.

Let’s get the Thrift definition file, that describes how Cassandra works, it’s interface/cassandra.thrift from the Cassandra source. Now we can use the Thrift compiler, to generate C# source files for Cassandra client: ./thrift -gen csharp cassandra.thrift. Output is in gen-csharp/Apache/Cassandra, just cd there and build it all: dmcs -r:Thrift -t:library -out:Apache.Cassandra.dll *.cs. Don’t forget to copy Thrift.dll there, or add the appropriate -lib:.

That should be it, we have all we need - a Thrift library (Thrift.dll) and a Cassandra client library (Apache.Cassandra.dll).

2011-09-24 17:42

C SharpMonoCassandraThrift

Web Server In 50 Lines

Web servers are useful, but not always you want or can install and configure one. Here’s a self contained, functional, multi-threaded and extensible web server in just few lines of C#.

 1 using System;
 2 using System.IO;
 3 using System.Net;
 4 using System.Text;
 5 
 6 
 7 class Server {
 8 
 9   static String root = "/home/you/web/root/";
10 
11   static String prefix = "http://localhost:8000/";
12 
13   static void Main(String[] args) {
14     using (var http = new HttpListener()) {
15       http.Prefixes.Add(prefix);
16       http.Start();
17       while (http.IsListening) {
18         var callback = new AsyncCallback(dispatch);
19         var result = http.BeginGetContext(callback, http);
20         result.AsyncWaitHandle.WaitOne();
21       }
22     }
23   }
24 
25   static void dispatch(IAsyncResult result) {
26     var http = (HttpListener)result.AsyncState;
27     var context = http.EndGetContext(result);
28     using (var responseStream = new MemoryStream()) {
29       var localPath = Path.Combine(root, context.Request.Url.LocalPath.Substring(1));
30       if (File.Exists(localPath)) {
31         using (var file = File.OpenRead(localPath)) {
32           file.CopyTo(responseStream);
33         }
34       }
35       else {
36         var text = Encoding.UTF8.GetBytes("404");
37         responseStream.Write(text, 0, text.Length);
38       }
39       context.Response.ContentLength64 = responseStream.Length;
40       responseStream.WriteTo(context.Response.OutputStream);
41       context.Response.OutputStream.Close();
42     }
43   }
44 }

So, you start up a HttpListener and pass every request it receives to dispatch(), there you can get a context that holds a HttpRequest and a HttpResponse, that you know and love, from there on, you’re free on how exactly you create the response, I check, if the requested URL can be mapped to a file, if so, I return that file, if not, I simply write “404”. That should give you a good idea, on how to expand this to match your needs.

Works quite well in my experience. One thing you have to be careful about, is exceptions in dispatch(), as they are in a new thread, you obviously won’t see them in Main().

2011-09-15 22:07

C Sharp.NETMono

Error Logging

One of the very first things I do, for a new ASP.NET application, is to setup a decent error logging. I want to know when and why my application crashes and I don’t want to relay on users accurately reporting every error. It’s quite easy to do this using HttpApplication.Error event:

  1 using System;
  2 using System.Configuration;
  3 using System.Data;
  4 using System.Data.SqlClient;
  5 using System.Security.Cryptography;
  6 using System.Text;
  7 using System.Web;
  8 
  9 
 10 public class Application : HttpApplication {
 11 
 12   private String Md5Sum(String message) {
 13     var bytes = Encoding.UTF8.GetBytes(message);
 14     var digest = (new MD5CryptoServiceProvider()).ComputeHash(bytes);
 15     var checksum = new StringBuilder();
 16     foreach (var b in digest) {
 17       checksum.Append(b.ToString("X2"));
 18     }
 19     return checksum.ToString();
 20   }
 21 
 22   private void LogError(String checksum, Int32 httpCode, String type, String message, String stack, String source, String url, String user, String machine, String os, String version) {
 23     var connectionString = ConfigurationManager.ConnectionStrings["DB"].ConnectionString;
 24     using (var connection = new SqlConnection(connectionString)) {
 25       connection.Open();
 26       var command = connection.CreateCommand();
 27       command.CommandText = @"
 28         UPDATE errors
 29         SET
 30           last_occurrence = current_timestamp,
 31           occurrences = occurrences + 1,
 32           http_code = @http_code,
 33           type = @type,
 34           message = @message,
 35           source = @source,
 36           url = @url,
 37           [user] = @user,
 38           machine = @machine,
 39           os = @os,
 40           version = @version
 41         WHERE
 42           checksum = @checksum;
 43       ";
 44       command.Parameters.Add("http_code", SqlDbType.SmallInt).Value = httpCode;
 45       command.Parameters.Add("type", SqlDbType.VarChar, 250).Value = type;
 46       command.Parameters.Add("message", SqlDbType.VarChar, 1000).Value = message;
 47       command.Parameters.Add("source", SqlDbType.VarChar, 100).Value = source;
 48       command.Parameters.Add("url", SqlDbType.VarChar, 150).Value = url;
 49       command.Parameters.Add("user", SqlDbType.VarChar, 50).Value = user;
 50       command.Parameters.Add("machine", SqlDbType.VarChar, 50).Value = machine;
 51       command.Parameters.Add("os", SqlDbType.VarChar, 100).Value = os;
 52       command.Parameters.Add("version", SqlDbType.VarChar, 100).Value = version;
 53       command.Parameters.Add("checksum", SqlDbType.VarChar, 32).Value = checksum;
 54       if (command.ExecuteNonQuery() == 0) {
 55         command.CommandText = @"
 56           INSERT INTO errors (checksum, http_code, type, message, stack, source, url, [user], machine, os, version)
 57           VALUES (@checksum, @http_code, @type, @message, @stack, @source, @url, @user, @machine, @os, @version);
 58         ";
 59         command.Parameters.Add("stack", SqlDbType.Text).Value = stack;
 60         command.ExecuteNonQuery();
 61       }
 62     }
 63   }
 64 
 65   public void Application_Error(Object sender, EventArgs e) {
 66     Server.ClearError();
 67     Response.Clear();
 68     var exception = Server.GetLastError().GetBaseException();
 69     var stack = exception.StackTrace;
 70     var checksum = Md5Sum(stack);
 71     var httpCode = ((HttpException)Server.GetLastError()).GetHttpCode();
 72     var httpMessage = Response.StatusDescription;
 73     var type = exception.GetType().ToString();
 74     var message = exception.Message;
 75     var source = exception.Source;
 76     var url = Request.Path;
 77     var user = Environment.UserName;
 78     //var user = Request.ServerVariables["AUTH_USER"];
 79     var machine = Environment.MachineName;
 80     var os = Environment.OSVersion.ToString();
 81     var version = Environment.Version.ToString();
 82     var dateTime = DateTime.UtcNow.ToString("o");
 83     var details = "";
 84     var developers = ConfigurationManager.AppSettings["Developers"].Split(';');
 85     //if (Array.IndexOf(developers, Request.ServerVariables["REMOTE_ADDR"]) > -1) {
 86     if (Array.IndexOf(developers, user) > -1) {
 87       details = String.Format(@"
 88         <p>[<b>{0}</b>: {1}]</p>
 89         <pre><code>{2}</code></pre>
 90         <hr/>
 91         <p>{3} ({4}), {5}<i> -- {6} @ {7} ({8}/{9})</i></p>
 92       ", type, message, stack, source, url, dateTime, user, machine, os, version);
 93     }
 94     var page = String.Format(@"
 95       <?xml version=""1.0"" encoding=""UTF-8""?>
 96       <!DOCTYPE html PUBLIC ""-//W3C//DTD XHTML 1.1//EN"" ""http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"">
 97       <html xmlns=""http://www.w3.org/1999/xhtml"" xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xsi:schemaLocation=""http://www.w3.org/MarkUp/SCHEMA/xhtml11.xsd"" xml:lang=""en"">
 98         <head>
 99           <title>{0} - {1}</title>
100         </head>
101         <body style=""font-size:smaller;"">
102           <p style=""text-align:center;color:white;background:orangered;""><b>{2}</b></p>
103           {3}
104         </body>
105       </html>
106     ", httpCode, httpMessage, checksum, details);
107     Response.Write(page);
108     Response.StatusCode = httpCode;
109     LogError(checksum, httpCode, type, message, stack, source, url, user, machine, os, version);
110   }
111 
112 }

You could put these methods directly into global.asax, but I prefer to compile and drop the DLL into /bin, or to save everything as a file with .cs extension into /app_code. Two later cases requires this one line in global.asax:

1 <%@ Inherits="Application" %>

It’s probably not a good idea to display error details to every user, just a checksum should be enough, so we’ll put users that need to see full errors into web.config:

1 <?xml version="1.0" encoding="utf-8"?>
2 <configuration>
3   <appSettings>
4     <add key="Developers" value="Alice;Bob" />
5   </appSettings>
6 </configuration>

One tricky thing about all this, is exceptions thrown from Application_Error() method. It would make sense, if these exceptions would show up as regular ASP.NET errors, but that’s not how it works in my experience. To show our own error page, we need to clear the error that resulted in Application_Error() being called, if we don’t call ClearError(), the original error will get displayed instead of our own, this also means that, if an exception gets thrown before ClearError(), you won’t see it. Situation with exceptions after ClearError() is a bit mysterious, on .NET I get a blank page, on Mono I get a regular ASP.NET error. Just know that, if something isn’t working as expected, first thing to try is to wrap everything in try-catch, to see if there’s any hidden exceptions.

Here’s the table definition, if you decide to use something similar:

 1 CREATE TABLE errors (
 2   id smallint IDENTITY NOT NULL,
 3   checksum char(32) NOT NULL UNIQUE,
 4   last_occurrence smalldatetime NOT NULL DEFAULT current_timestamp,
 5   occurrences int NOT NULL DEFAULT 1,
 6   http_code smallint NOT NULL,
 7   type varchar(250) NOT NULL,
 8   message varchar(1000) NOT NULL,
 9   stack text NOT NULL,
10   source varchar(100) NOT NULL,
11   url varchar(150) NOT NULL,
12   [user] varchar(50) NOT NULL,
13   machine varchar(50) NOT NULL,
14   os varchar(100) NOT NULL,
15   version varchar(100) NOT NULL,
16   CONSTRAINT PK_errors PRIMARY KEY CLUSTERED (id)
17 );

2010-03-09 02:00

C Sharp.NETMonoASP.NET

Computing MD5/SHA256/SHA512 Checksum

 1 using System;
 2 using System.Security.Cryptography;
 3 using System.Text;
 4 
 5 class Test {
 6 
 7   public static int Main(String[] args) {
 8     if (args.Length != 2) {
 9       return 1;
10     } else {
11       HashAlgorithm algo;
12       switch (args[0].ToUpper()) {
13         case "MD5":
14           algo = new MD5CryptoServiceProvider();
15           break;
16         case "SHA256":
17           algo = new SHA256CryptoServiceProvider();
18           break;
19         case "SHA512":
20           algo = new SHA512CryptoServiceProvider();
21           break;
22         default:
23           return 2;
24       }
25       Console.WriteLine(ComputeChecksum(algo, args[1]));
26       return 0;
27     }
28   }
29 
30   static public String ComputeChecksum(HashAlgorithm algo, String message) {
31     var bytes = Encoding.UTF8.GetBytes(message);
32     var digest = algo.ComputeHash(bytes);
33     var checksum = new StringBuilder();
34     foreach (Byte b in digest) {
35       checksum.Append(b.ToString("x2"));
36     }
37     return checksum.ToString();
38   }
39 
40 }

2010-01-24 22:24

.NETMonoC Sharp

URL Mapping

Here’s full, working and as simple as possible example on how to get pretty URLs with .NET/Mono.

First, here’s all the code:

 1 using System;
 2 using System.Web;
 3 using System.Web.Routing;
 4 
 5 
 6 class RouteHandler : IRouteHandler {
 7 
 8   Type PageType;
 9 
10   public RouteHandler(Type pageType) {
11     PageType = pageType;
12   }
13 
14   public IHttpHandler GetHttpHandler(RequestContext requestContext) {
15     var routeData = requestContext.RouteData.Values.Values;
16     string[] args = new string[routeData.Count];
17     routeData.CopyTo(args, 0);
18     Page page = (Page)Activator.CreateInstance(PageType, args);
19     return page;
20   }
21 
22 }
23 
24 
25 public class Global : HttpApplication {
26 
27   protected void Application_Start() {
28     RouteCollection r = RouteTable.Routes;
29     r.Add(new Route("foo", new RouteHandler(typeof(Foo))));
30     r.Add(new Route("bar/{id}", new RouteHandler(typeof(Bar))));
31   }
32 
33 }
34 
35 
36 class Page : System.Web.UI.Page {
37 
38   override protected void OnLoadComplete(EventArgs e) {
39     if (Request.HttpMethod == "POST") {
40       Post();
41     } else {
42       Get();
43     }
44   }
45 
46   virtual protected void Get() {
47     throw new NotImplementedException();
48   }
49 
50   virtual protected void Post() {
51     throw new NotImplementedException();
52   }
53 
54 }
55 
56 
57 class Foo : Page {
58 
59   override protected void Get() {
60     Response.Write("FOO!");
61   }
62 
63 }
64 
65 
66 class Bar : Page {
67 
68   string Id;
69 
70   public Bar(string id) {
71     Id = id;
72   }
73 
74   override protected void Get() {
75     Response.Write("BAR" + Id + "!");
76   }
77 
78 }

Save that as global.cs and compile it with this, if you’re using Mono:

1 gmcs -debug+ -r:System.Web,System.Web.Routing -t:library global.cs

Or this, if you’re using .NET:

1 csc /debug+ /r:System.Web.Routing.dll /t:library global.cs

Drop the resulting global.dll in Bin directory on the web server. And to wire it up, you’ll need this global.asax:

1 <%@ Inherits="Global" %>

Also, your web.config should look something like this:

 1 <?xml version="1.0"?>
 2 <configuration>
 3   <system.web>
 4     <customErrors mode="Off" />
 5     <httpModules>
 6       <clear />
 7       <add name="UrlRoutingModule" type="System.Web.Routing.UrlRoutingModule, System.Web.Routing, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
 8     </httpModules>
 9   </system.web>
10 </configuration>

And that should be it! Try visiting /foo and /bar/123 on the web server. I successfully tested this with xsp2, lighttpd and IIS 6.0, just remember that, for this to work, web server needs to send all requests to CGI/ISAPI module, xsp2 always does this, in lighttpd, simply send all requests to fastcgi-mono-server, in IIS, send everything to aspnet_isapi.dll using “wildcard application maps” setting.

2010-01-11 00:13

.NETC SharpMonoASP.NET

Type-Safe Collection of Diverse Types

So you have a couple of different objects (say, Foo and Bar), and would like to store them all in some sort of collection (Col), but in a type-safe way.

Here’s two solutions:

  1 using System;
  2 using System.Collections;
  3 using System.Collections.Generic;
  4 
  5 
  6 // required only for Col2
  7 public interface IItem {
  8 
  9   void Print();
 10 
 11 }
 12 
 13 
 14 public class Foo : IItem {
 15 
 16   public String Text;
 17 
 18   public Foo(String text) {
 19     Text = text;
 20     Print();
 21   }
 22 
 23   public void Print() {
 24     Console.Write("Foo:" + Text + "; ");
 25   }
 26 
 27 }
 28 
 29 
 30 public class Bar : IItem {
 31 
 32   public List<String> List;
 33 
 34   public Bar(List<String> list) {
 35     List = list;
 36     Print();
 37   }
 38 
 39   public void Print() {
 40     Console.Write("Bar:");
 41     foreach (String item in List) {
 42       Console.Write(item + ";");
 43     }
 44     Console.Write(" ");
 45   }
 46 
 47 }
 48 
 49 
 50 // stores objects as they are, without casting
 51 public class Col1 {
 52 
 53   private List<Foo> fooList = new List<Foo>();
 54 
 55   private List<Bar> barList = new List<Bar>();
 56 
 57   private List<Char> order = new List<Char>();
 58 
 59   public void Add(Foo item) {
 60     fooList.Add(item);
 61     order.Add('f');
 62   }
 63 
 64   public void Add(Bar item) {
 65     barList.Add(item);
 66     order.Add('b');
 67   }
 68 
 69   public void Print() {
 70     var fooEnu = fooList.GetEnumerator();
 71     var barEnu = barList.GetEnumerator();
 72     foreach (Char list in order) {
 73       switch (list) {
 74         case 'f':
 75           fooEnu.MoveNext();
 76           fooEnu.Current.Print();
 77           break;
 78         case 'b':
 79           barEnu.MoveNext();
 80           barEnu.Current.Print();
 81           break;
 82         default:
 83           throw new Exception();
 84       }
 85     }
 86   }
 87 
 88 }
 89 
 90 
 91 // stores objects cast to interface
 92 public class Col2 {
 93 
 94   private List<IItem> list = new List<IItem>();
 95 
 96   public void Add(IItem item) {
 97     list.Add(item);
 98   }
 99 
100   public void Print() {
101     foreach (IItem item in list) {
102       item.Print();
103     }
104   }
105 
106 }
107 
108 
109 // test collection with random data
110 public class Test {
111 
112   public static void Main() {
113     Col1 col = new Col1();
114     //Col2 col = new Col2();
115     Random rng = new Random();
116     for (Int16 i = 0; i < rng.Next(3, 10); i++) {
117       if (rng.Next(2) == 0) {
118         col.Add(new Foo(i.ToString()));
119       } else {
120         List<String> list = new List<String>();
121         for (Int16 i2 = 0; i2 < rng.Next(1, 4); i2++) {
122           list.Add(i.ToString() + i2.ToString());
123         }
124         col.Add(new Bar(list));
125       }
126     }
127     Console.WriteLine();
128     col.Print();
129     Console.WriteLine();
130   }
131 
132 }

2010-01-03 01:50

.NETMonoC Sharp

| Archive | RSS | E-mail | Twitter | Alvis Mikovs