When I firstly heard about Crystal language, I got really exited. It fixes exactly four problems I sometimes have with Ruby; types, speed, memory consumption and compilation to machine code. After many months silently following its development I decided to try it out for a simple program I need — a simple server standing between Google Maps API and my application to catch the geocoding responses in local Redis instance not to hit the limit imposed on using the service (plus, Google suggests you to do it anyway).
Before we start with the code let’s look at the requirements. We need to query Google Maps API for geocoding of addresses. That means parsing JSON. We also need to save the latitude and longitude for the objects in question in local Redis instance. That means connecting to and using Redis from Crystal. And finally we need to expose this as a local service, so we need to run our program as a server waiting for our geocoding requests.
For this problems I decided to go with HTTP::Client and JSON libraries included in Crystal, together with crystal-redis
client and Kemal for the server bits. I chose all of that after few minutes of googling, so make no conclusions out of it.
Let’s call our program geoserver
. This is how the shards.yml
would look like:
$ cat shard.yml name: geoserver version: 0.1 dependencies: redis: github: stefanwille/crystal-redis version: ~> 1.5.1 kemal: github: sdogruyol/kemal
Shards can fetch and install the dependencies for us. To use the libraries in question, this would be the require calls at the top:
require "http/client" require "kemal" require "json" require "redis"
Now we can slowly start implementing our program. The core of it all is to handle responses from Google Maps API, so let’s look at the success and error responses we have to deal with:
{ "results" : [ { "address_components" : [ { "long_name" : "Opava", "short_name" : "Opava", "types" : [ "locality", "political" ] }, { "long_name" : "Opava District", "short_name" : "Opava District", "types" : [ "administrative_area_level_2", "political" ] }, { "long_name" : "Moravian-Silesian Region", "short_name" : "Moravian-Silesian Region", "types" : [ "administrative_area_level_1", "political" ] }, { "long_name" : "Czech Republic", "short_name" : "CZ", "types" : [ "country", "political" ] } ], "formatted_address" : "Opava, Czech Republic", "geometry" : { "bounds" : { "northeast" : { "lat" : 49.9939829, "lng" : 17.9964488 }, "southwest" : { "lat" : 49.8477767, "lng" : 17.7905329 } }, "location" : { "lat" : 49.9406598, "lng" : 17.8947989 }, "location_type" : "APPROXIMATE", "viewport" : { "northeast" : { "lat" : 49.9939829, "lng" : 17.9964488 }, "southwest" : { "lat" : 49.8477767, "lng" : 17.7905329 } } }, "place_id" : "ChIJm8CGORPYE0cRvzf4rjv_0RQ", "types" : [ "locality", "political" ] } ], "status" : "OK" }
And for the error response we might get the following:
{ "error_message" : "You have exceeded your daily request quota for this API. We recommend registering for a key at the Google Developers Console: https://console.developers.google.com/apis/credentials?project=_", "results" : [], "status" : "OVER_QUERY_LIMIT" }
Exceeding the quota can happen sooner than you think and that’s why we are building this! And if you are wondering what kind of place Opava is, that’s the city I was born in.
So let’s start with some Crystal!
module GoogleMapsApi class Location JSON.mapping({ lat: Float64, lng: Float64, }) end class Geometry JSON.mapping({ location: Location }) end class Result JSON.mapping({ geometry: Geometry }) end class SuccessResponse JSON.mapping({ results: Array(Result) }) end class ErrorResponse JSON.mapping({ error_message: String }) end ...
I am splitting every part of the responce in its own class so we can nicely work with the respective objects. At the top we are getting either SuccessResponse
or ErrorResponse
. As you can see we can use JSON.mapping
to specify our objects mapping to JSON format. This is very convenient as we can now call methods to_json
or from_json
with all parts of the response.
To simplify everything we also don’t need to specify all JSON fields from the API. We are gonna implement the minimal representation that gets us what we need. If we want though we can list all the other fields here and even use JSON::Any
type if we don’t need to map the values to anything concrete.
With what we have we can parse the response and the latitude and longitude of our address:
r = GoogleMapsApi::SuccessResponse.from_json("...copy the response here...") r.results[0].geometry.location.lat # => 49.9407
This sounds promising. Let’s implement a client that can query the Google Maps API for us.
module GoogleMapsApi class Client class ServerError < Exception; end Host = "maps.googleapis.com" def initialize @http_client = HTTP::Client.new(Host, ssl: true) end def get(location : String) response = @http_client.get("/maps/api/geocode/json?address=#{location}") process_response(response) end private def process_response(response : HTTP::Client::Response) case response.status_code when 200..299 # For cases when "status" is one of the following: # ZERO_RESULTS # OK return SuccessResponse.from_json(response.body) \ if ["OK", "ZERO_RESULTS"].includes?(Response.from_json(response.body).status) # For cases when "status" is one of the following: # OVER_QUERY_LIMIT # REQUEST_DENIED # INVALID_REQUEST raise Client::ServerError.new(ErrorResponse.from_json(response.body).error_message) when 400 raise Client::ServerError.new("400: Server Not Found") when 500 raise Client::ServerError.new("500: Internal Server Error") when 502 raise Client::ServerError.new("502: Bad Gateway") when 503 raise Client::ServerError.new("503: Service Unavailable") when 504 raise Client::ServerError.new("504: Gateway Timeout") else raise Client::ServerError.new("Server returned error #{response.status_code}") end end ...
Our client can’t authenticate you, but will be sufficient if we are using only the limited number of requests without the API key.
The only two things to notice here (especially if you have some Ruby background) are types specifications in method signatures and the need to initialize instance variables in initialize
method (as we do with @http_client
).
Moving on the last piece of our example program is the server part. Here is a “1 minute Crystal & Redis course”:
puts "Connect to Redis" redis = Redis.new puts "Delete foo" redis.del("foo") puts "Set foo to \"bar\"" redis.set("foo", "bar") puts "Get the value of foo:" redis.get("foo")
With our new Redis skills let’s implement our API in Kemal. This is the idea:
get "/:address" do |env| address = env.params.url["address"] ... # retrieve the address geolocation from Redis if available # otherwise query the address from Google Maps API and save to Redis # return the location as JSON object end Kemal.run
Kemal will now automatically start the server and will listen for our requests in the format of /address-we-want-to-geocode
.
The last thing is to put all the pieces together:
get "/:address" do |env| address = env.params.url["address"] env.response.content_type = "application/json" location = redis.get(address) if location response = { location: location } else begin record = GoogleMapsApi::Client.new.get(address) if match = record.results[0] location = match.geometry.location else location = GoogleMapsApi::Location.from_json(%({"lat": 0.0, "lng": 0.0})) end redis.set(address, location.to_json) response = { location: location } rescue ex : GoogleMapsApi::Client::ServerError response = { error: ex.message } end end response.to_json end Kemal.run
Again one thing to notice here is the different syntax for handling exceptions. Apart from that the whole experience felt very much like Ruby and that’s what I personally love about Crystal. It has the features of so much praised Golang with Ruby-like syntax and I am exited about its future.
Where are the tests?
Feel free to look at the spec folder here: https://github.com/strzibny/hipparchus