1) Pay Twilio and Tropo for the use of their APIs, for roughly a penny per message.

2) Send an email directly to their carrier’s mail-to-SMS gateway – a free service most carriers provide, which will translate your email into an SMS sent to their subscribers’ phone

Twilio and Tropo are about as easy as it gets to integrate (REST-like APIs that return JSON/POX), but you pay for every single message and that can add up quickly. So #2 (mail-to-SMS gateways) starts to look pretty good.

But there’s some problems with mail-to-SMS gateways:

1) Each carrier has their own format. For example, T-Mobile’s gateway is number@tmomail.net (for the US), AT&T’s is number@txt.att.net (for most of their customers), and Verizon is number@vtext.net (unless you used to be with Alltel – then its number@message.alltel.com). A big list of mail-to-SMS-gateways is available on Wikipedia.

Bonus:some carriers want the international code prepended (the 1 in front of the number, for the US), some carriers don’t. (e.g., T-Mobile does, AT&T doesn’t)

2) Some carriers require authentication (Sprint) or don’t have a mail-to-SMS gateway at all (Google Voice).

3) If the user ports their number to a different carrier you’ll have to detect it and react accordingly.

So you’ll have to figure out what carrier they’re using in order to direct the email and meet whatever requirements – which usually means asking the user to choose their carrier from a list.

There is a better way! What if we could detect the carrier automatically?

It turns out you can. The lovely folks at CloudVox made an API that does exactly that – you give it a phone number, and it tells you the carrier and if its a cell-phone or landline. Its 100% free, with no API keys/credentials required.

Its hosted at digits.cloudvox.com, along with example calls/responses (full docs are here). It speaks JSON over HTTP.

This gives us a much better user experience:

1) User inputs their number, no carrier dropdown needed

2) We use CloudVox’s API to determine their carrier

3) Try to send them an SMS, and have them confirm ownership in the normal way (i.e., enter a confirmation code)

This will cover most people, and is free outside of the development time. If there was an error (i.e., we couldn’t find the number with CloudVox’s API, or we did but the carrier doesn’t support mail-to-SMS), we can fall back to Twilio or Tropo. We’re still paying a few cents here and there, but the default case is free.

Here’s some example code in PHP, for querying CloudVox’s API:

<?php
	/* Finds the carrier for the given phone number using CloudVox's API. Returns the carrier name as a string on success, or false if the carrier couldn't be determined */
	function findCarrier($phoneNumber)
	{
		$carrier = false; // will contain the carriers name as a string if we are able to look it up
 
		// attempt to lookup the number with the CloudVox API
		$requestUrl = 'http://digits.cloudvox.com/' . $phoneNumber . '.json';
		$apiResponse = file_get_contents($requestUrl); // Note: needs code to handle the error case of file_get_contents
 
		if ($apiResponse)
		{
			$responseJson = json_decode($apiResponse);
			if (isset($responseJson->allocation) && isset($responseJson->allocation->carrier_name))
			{
				$carrier = $responseJson->allocation->carrier_name;			
			}
		}
		return $carrier; // will be false or the name of the carrier
	}
 
	if (isset($_GET['phoneNumber']))
	{
		// user has submitted a form via GET, containing a field called phoneNumber
 
		// collect the phoneNumber and remove any non-numeric chars
		$phoneNumber = preg_replace('#[^0-9]#','',strip_tags($_GET['phoneNumber'])); 
		$carrier = findCarrier($phoneNumber);
 
		if ($carrier)
		{
			if (stripos($carrier, 'verizon') !== FALSE)
			{
				sendWithVerizon($phoneNumber);
			}
			else if (stripos($carrier, 'at&t') !== FALSE)
			{
				sendWithATT($phoneNumber);
			}
			else
			{
				// other carrier. you'd really want to fall back to Twilio/Tropo here, and/or log the carrier so you can add it later
				echo "Unknown Carrier: {$carrier}";
			}
		}
		else
		{
			// send SMS via Twilio or Tropo instead
		}
	}
	else
	{
		// show the form
		echo '<form>' . 
			 '<label for="phoneNumber">Phone Number</label>' .
			 '<input type="text" name="phoneNumber" />' .
			 '</form>';
	}
?>

Consider the following code:

function fetchUrl($url){
     $ch = curl_init();
     curl_setopt($ch, CURLOPT_URL, $url);
     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
     curl_setopt($ch, CURLOPT_TIMEOUT, 20);
 
     $retData = curl_exec($ch);
     curl_close($ch); 
 
     return $retData;
}

$profile_id = "1234567890";
 
$data['feed_data'] = fetchUrl("http://graph.facebook.com/{$profile_id}/feed");

Before the recent changes, this was all you needed to load a Facebook page’s feed. You could load a public page’s feed by simply pulling from graph.facebook.com/{profile_id}/feed. With the recent changes you just have to make a small modification and you’re good to go.

Here is the updated code:

$profile_id = "1234567890";		
 
//App Info, needed for Auth
$app_id = "0001234567890";
$app_secret = "abc123ebf123f3g5g6j";
 
//retrieve auth token
$authToken = fetchUrl("https://graph.facebook.com/oauth/access_token?type=client_cred&client_id={$app_id}&client_secret={$app_secret}");
 
$data['feed_data'] = fetchUrl("https://graph.facebook.com/{$profile_id}/feed?{$authToken}");

Be sure to use https instead of http, needed for OAuth. You’ll also need to create a simple Facebook app to get an App ID and Secret key. Once done, just swap out the variables in the above script for the proper values and you’re off to the races!

The Challenge

Given 2 text files, each containing approximately 1500 anonymous IP addresses, we were to determine how many resolved to each US state. Separate counts were to be maintained for each list. The data was needed for a time sensitive report, and the clock was ticking.

The Tools

There are several freely available SQL databases which correlate IP addresses to their geographic location, such as this one maintained by iplocationtools.com. If we’d been building a web app setting up a local instance would have made sense, but at 4.1 million rows it was going to take too long to get a local instance running.

What we needed was a web service to do the lookups for us. In the past, we’d been unable to find a free web service to fulfill this role, as most will not provide data beyond a country level. But today we were lucky, and Google quickly turned up the perfect solution: a free REST web service running the very same database from iplocationtools.com. Its exactly what it says on the tin: a free IP geolocation webservice. It gave us even more accuracy than we needed (cities and zipcodes!), and even better it was running on Google’s App Engine, so we knew it could handle a heavy load.

With the primary problem solved, the only thing needed was a simple PHP script to read in the input files, query the web service for each IP, parse the JSON results, and maintain a total for each state.

The Solution

Thanks to PHP5′s excellent JSON support, the script to interact with the web service was trivial. I won’t bore you with the file I/O and tallying, but here’s the function which converts the IP address to its state.

<?php
	function getStateFromIp($ip)
	{
		$resp = json_decode(file_get_contents('http://freegeoip.appspot.com/json/' . urlencode($ip)));
		if (!$resp->status)
		{
			// lookup failed
			return false;
		}
		else
		{
			return $resp->regioncode;		
		}
	}      
?>

Take note of the the json_decode function. If you’re not already familiar with it I’d highly recommend taking a few minutes to familiarize yourself with PHP5′s JSON functions. JSON is becoming a de facto standard for web services, so being able to painlessly convert to and from JSON in PHP is critical.

Finally, we simply used print_r to dump the array containing the per state totals, and used Monarch to pull the data into an Excel file which we then delivered to the customer. (Note: under normal circumstances we would have reached a more elegant solution, but given the time constraints brute force was the name of the game.)

Conclusion

On-demand web services are fundamentally changing the way we interact with data. Projects which were formerly large, slow, and expensive can now be completed in a matter of hours or days for a fraction of the cost. Perhaps most importantly, anyone can get involved.

Its an exciting time, and we’re happy to be a part of it all. Thanks for reading!

If your company needs help with a custom dataset contact us today. Even if you’re not sure exactly what to ask for, we can help fill in the blanks.