I've noticed it can be hard to find the mochiweb docs online, so I compiled them and I'm now hosting them at mochiweb documentation. Feel free to ask in the comments if you'd like me to host anything else at http://dawsdesign.com/drupal/man-pages.
While thinking about ErlFS, I realized I'd need at least two subsystems. One for finding a node with the data I am looking for and one to actually store that data. The first will be covered by Chordial. For the storage portion, I was originally going to use a homegrown method where I would just store the data in files named after their key. I decided I should explore the various possiblities and so I did a little research and decided to take a crack at writing a port driver for Erlang. Thanks go to Kevin Smith for his article on writing linked-in drivers for Erlang.
There are several way to write an Erlang driver. One way is to spawn a process and communicate over a pipe to your program and another is to bring your program in the same memory space and communicate directly with in-memory vectors. The first is safe; if your program crashes, your Erlang application can recover using standard OTP principles, such as supervisors. The in-vm-memory model is much more dangerous; if your program crashes, the whole Erlang VM crashes with it. This method, however, is much faster, as it does not have to copy memory and then have the OS switch context for each call. We will be covering this method.
Writing your first Erlang driver can be very difficult as there are very limited examples and the documentation can be very obscure. In this article, I will walk you through writing one for a common purpose-- writing to a database.
All sources can be found at the project's GitHub page.
First let's start with a simple OTP build directory structure. I'll create a new folder called erl_bdb_store and in that make the following folders: ebin, include, priv, and src. This is what it should look like:
Next, we'll need a couple simple makefiles. Please note that these currently only compile under linux (tested on Ubuntu). The first one just goes to the /priv and /src directories and runs the Makefiles in each.
The next one compiles the C source (the meat-and-potatoes of the driver) into a shared library. This is different on each operating system and will need to be changed for each. I will do this sometime in the future.
The erlang Makefile is simple
./src/Makefile
Now we will make our Erlang wrapper for the driver. It is a standard OTP gen_server behavior callback.
./src/bdb_store.erl
The two important parts are the init function and the handle_call function. The init function loads and starts the driver.
The handle_call function communicates with the driver to put, get and delete records from the database file. This just forwards and translates the commands to the actual driver, where all of the real work happens:
Notice the primary difference between each call is the first byte of each Message binary. This byte is inspected inside the driver and there it is determined which function to perform:
The C code does most of the work. It handles creating all of the error messages and performing all of the commands the Erlang source communicates to it.
Let's take a look at the C source piece-by-piece.
First we'll create a header file with settings and function prototypes.
./priv/bdb_drv.h:
At the top we include necessary Erlang headers followed by a couple standard C headers and then the Berkeley DB header. Next we define the path where we want our database to store data, and then we define constants for the byte values of the commands sent to the driver from the Erlang VM. The _bdb_drv_t struct is very important, as we cannot use global variables and so we must keep our state in a struct. The reason is that the Erlang VM can and will start a new instance of the driver many times for performance and we want to keep our state localized. The last entries in the header file are our function prototypes.
Now to our C implementation file (./priv/bdb_drv.c).
Include our header file:
Specify the callbacks we will be implementing:
This array defines the callback functions that will be invoked at various points by the Erlang VM. In this example, we are implementing start, stop and outputv. This means that when the Erlang VM starts an instance of our driver, it will call our function start, when it sends a message, it will call outputv and when it stops the instance, it will call stop. outputv is only used if output is not defined in the array. output uses a buffer whereas outputv does not, meaning there is no overhead in copying the data, so it is faster. See driver_entry documentation for more detail.
Next is boilerplate which tells the VM which struct to use as the state holder:
Here, we define the start function and open up the database for reading and writing.
If Berkeley DB returns an error, it is propagated to the Erlang VM as a typical {error, Reason} tuple.
The stop function closes the database and releases the driver:
outputv will serve as our entry point when a message is sent to the driver:
It interprets the first byte of the message to determine which function to call.
As it's name implies, the put function inserts a record into the database:
The function grabs the first 20 bytes after the first command byte and uses it as the key. This leaves room for 160 bits-- enough for a SHA1 hash. All of the remaining bytes are used as the value to be stored. This function returns the atom ok or the typical {error, Reason} tuple.
The get function is similar to put, except we have to free the memory allocated by Berkeley after the record has been retrieved:
First we call an Erlang function named driver_alloc_binary (see erl_driver#driver_alloc_binary) which keeps a reference count, starting with 1. We then return the value with driver_output_term which increases the reference count to 2 and then call driver_free_binary which brings it to 1 again. When the Erlang VM has finished with it, it will also call driver_free_binary which brings the reference count to 0 and it will be freed from memory.
The del function deletes a record and is the most simple of the database functions:
Finally we have our catch-all function which will be called if an unrecognized command byte is sent. It returns the tuple {error, unkown_command}:
Let's compile it and jump into an Erlang shell to test it out:
Now, let's start the gen_server we wrote and perform a couple tests:
That's it for the walkthrough. If you have any questions, please post a comment or use the contact form. You can also email me at the email address listed in the source. I hope it helps you on a future project! Please let me know if it does!
All sources can be found at the project's GitHub page.
![]()
I've been stalled with ErlFS for a while now, but my interest has been flared up due to someone cloning my SVN repo into GitHub, where it ended up on the front page for interesting projects.
So, I had actually been scheming a bit and was thinking about using a distributed hash table instead of a mesh network, due to the ability to scale massively. This is one of the goals of ErlFS, so it seemed natural.
I'm dead set on it now and I've started a project called chordial, which is my open source _interpretation_ of MIT's chord algorithm. Once it is finished, it should greatly simplify ErlFS and I will likely start from scratch.
You can find chordial at http://github.com/dawsdesign/chordial/.
I can currently add multiple nodes to the ring, but I am having some issues with the finger function on the nodes. It will just take some time to get sorted out, but I am feeling great about this project and it is going quite well.
If you want to get your manager to try out Erlang, but they are hesitant (probably an understatement) you might be able to get some leverage if you use a relational database rather than mnesia. So, here's a quick sample of how to connect to and query a MySQL database.
First, you'll need to install the MySQL ODBC driver if you haven't already. You can find it at http://dev.mysql.com/downloads/connector/odbc/5.1.html. Then you'll need a database to connect to and a table to query. Here's the SQL script I used to create my test database:
And here's the code to connect and query in erlang:
This is what my interactive session looks like:
It's that simple. Post a comment if you want more samples or description.
Erlang docs: http://www.erlang.org/doc/apps/odbc/index.html
I came across the most impressive Erlang tutorial I have seen yet. Check it out. Now.
I'm working on a tile editor in Flash CS4 and opted to use the TileList class for the palette. Once a swatch is selected, I need to create a copy of that swatch and attach it to the stage. I wasn't quite sure how polymorphism in AS3 worked but I ended up with this little snippet:
Let me know if it helps!
I recently got an iPod Touch and it is the best gadget I've ever had. One of the features of the powerful little machine is the location service. This service uses the wifi radio to listen for wifi beacons, the signal an access point sends out every second or two to let clients know the AP is there. When you refresh your list of wireless networks, it gathers these signals to make the list. Anyway, the utility of this feature is actually quite extensive. You can use it to find nearby restaurants, movie showtimes, and make getting directions a breeze. Unfortunately in my home area, there is no coverage.
I began searching and found an article on MacRumors that mentioned Apple uses a company called Skyhook to provide their database of AP MAC addresses and their coordinates http://www.macrumors.com/2008/01/23/skyhooks-wi-fi-location-technology-f.... Skyhook, according to their website, keeps a database of MAC addresses and associates them with their GPS location. Using this database is claimed to be accurate within 10-20 Meters.
After visiting Skyhook's website, I found the form where you can add your Wifi Access Point to their database. The page is at http://www.skyhookwireless.com/howitworks/submit_ap.php. Unfortunately it can take up to a few weeks to have the data updated with all of their partners. I will repost here if it ends up working for me.
P. S. If you can't find your MAC address, this article might help http://getsatisfaction.com/skyhookwireless/topics/how_do_i_find_the_mac_....
UPDATE 12/22/2008
It worked! Now when I push the current location icon in Google Maps it centers right on my street. 12 days later ain't bad.
This was another feature that should have been simple, but just wasn't. This Google Maps extension allows you to use circle overlays. It is really a GPolygon with 45 points, but it looks like a circle. Here is a sample. You can download the source here.
js/gmaps.CircleOverlay.js:
circle.html:
At work, I've been tasked with a neat little project where a user can select a list of users represented by markers on a Google map by editing a polygon to surround the markers with. However, to my dismay, I could not find any methods in the API to see if a polygon contains a coordinate. I also had a hard time finding an algorithm so that I could implement it. I later learned this type of algorithm is called Point in Polygon (PiP).
First I tried the winding number algorithm (http://en.wikipedia.org/wiki/Point_in_polygon) but failed. I didn't put too much time into it since it is so CPU-intesive anyway. Then I came across an article on MSDN (http://msdn.microsoft.com/en-us/library/cc451895.aspx) which I adopted into Javascript. I extended the GPolygon class with a new method called containsLatLng, just like the GLatLngBounds class. The difference is that the method in the GLatLngBounds class will only get the 4 coordinates of the bounding box, which are the farthest points in the polygon, and see if the coordinate lies within that box. My method, however, does true collision detection to see if the point lies within any shape polygon with as many sides as you want. It worked out perfectly.
Without any further ado, here is the code [js/gmaps.polygon.containsLatLng.js]:
And here is the sample usage [index.htm]:
Sample: http://dawsdesign.com/Samples/GIS/
Download: http://dawsdesign.com/Samples/GIS/GIS.zip
This is a short video I put together. It shows how to use emacs to generate an Erlang gen_server to greet a client. It also demonstrates Erlang's hot code-loading capabilities.
Erlang/OTP gen_server demo from Matt Williamson on Vimeo.
Really high quality at http://dawsdesign.com/video/video-gen_server_demo