Everyone Benefits When Small Data Goes Big

Gadi Ben-Yehuda govtech panelIt is advantageous for everyone when people share certain data with one another and with the public.

For decades, urbanites have been tuning in to local radio stations before their morning commute to hear the traffic report.  Based on static cameras at strategic locations, as well as helicopters and tips from motorists or passengers, traffic reports are perfect examples of data that give rise to decisions.  Holland Tunnel is blocked? Route around it.  Pile up on the Inner Loop? Exit early.  But in 2008, drivers got a whole new way to see traffic in real time.

That year, Waze was released—a mobile app that used information from the cellphones of people who downloaded it, sharing their location and speed with other people who were using the same application to create a crowd-sourced traffic report.  By sharing the information on their phone, everyone benefited.  The “small data” coming from single users was anonymized, amalgamated, and the turned into Big Data that was visualized as a traffic map.

This is only one example of this phenomenon, but in the past few years, more types of data have begun going small and are now breaking into the mainstream.  Five areas seem especially ripe for people to share anonymized data with one another and with their community for the purpose of helping everyone realize the benefits that are normally associated with big data.  If people are willing to share basic information—voluntarily given and thoroughly anonymized—about their health, transportation, energy, finance, and the sharing economy, they will benefit from the insights that only big data analytics can offer.

The Benefits of Big Data

What are those benefits? The most succinct answer is also the most abstract, but it is: higher quality services, and better individual decision-making, at a lower cost.  That’s it.  The promise of analyzing big data (which presupposes that there is big data to analyze) is that it will result in people and organizations making better decisions, enjoying better services, and saving money. How that applies in each of the five areas is addressed below.

Five Examples of Small Data

Wearable technology may be flashiest way that data is entering people’s lives, but it is hardly the only way:

  • Financial data has been on an upswing for years, as services that were once available only for a fee (like Quicken or MSMoney) are now free (through MintMVelopesBuxfer, orReadyForZero).
  • Vehicle and Transportation Data: information from cars’ engine computers can be collected and understood through devices like Automatic or Bluetooth accessories for smartphones, and transportation data can be shared through Waze or Inrix.
  • Energy data can be collected, and immediately used to reduce energy costs, but a number of consumer devices.  Further, more cities are deploying smart grids, so that households can see exactly the same information that the utilities do, and in real-time.
  • Health data is also going mainstream, as wearables are being joined by smartphone accessories that replace many separate, commonplace home health devices—like a scales and thermometers.

Put together, these five areas begin to paint a picture not only of an individual’s life, but of the life of a community.  They can begin to help communities assess and then address their own needs both in terms of routine events, and by spotting and rectifying anomalies as they begin to occur, rather than when they have already spiraled out of control.

Protecting Privacy while Maintaining Utility

Though the benefits of sharing data are great, before any information is collected, the privacy of all participants must be assured.  Famously, three seemingly innocuous data can be used to identify 87% of US adults: gender, ZIP code, and birthdate.  That this can be done should not dissuade people from seeking the benefits of turning their small data into Big, but rather should reinforce the notion that only the data that actually helps should be collected.

For most applications, as an example, birthdate is superfluous information; at best only the birth year, or even better, a range of birth-years, would suffice to help people derive meaning, and thus benefit, from the data.  Further, ZIP codes could be replaced by other types of location data, for example, distance from major urban centers—e.g. “within a 5-mile radius of certain coordinates.”  Gender is unlikely to be needed for any of the applications discussed.

In each area, then, care must be taken both by individuals and whatever organization, public or private, that amalgamates the small data into large data sets to ensure that no individual is likely to surrender their personal identity.

The Difference between Data and Insights

The data, by themselves, are meaningless.  They are dots without connection.  The ability to make those connections are what makes the data so powerful, which is why government agencies have so many restrictions on collecting certain kinds of data.  What is important, however, is not the data but rather the insights to which they give rise, and that is why it is not necessary that government agencies themselves (as opposed to private-sector organizations) collect or even have access to the data.

An easy example of government agencies using the benefits of data without collecting, or even having access to it, is Facebook ads.  Users share information with Facebook, and government agencies make use of that data by crafting ads that appear only to specific demographics.  For the five topics listed below, government agencies could equally as easily work with extant companies or with companies yet to be founded that will exist solely to provide agencies with the insights discernible through analysis of small-data-made-big.

Small Data Goes Big for Personal and Public Good      

People have always collected data on these five areas of their lives, they just haven’t called it “collecting data.”  Balancing a checkbook and tracking a household’s finances are exercise in data analytics.  Charting the course of a child’s fever is an exercise data collection coupled, perhaps, with emergency management.  Listening to the traffic report in the morning and altering one’s route to work is data collection to power real-time decision making.

What makes the current situation different than, say, 20 years ago, are three elements: first, our ability to gather and store data has increased dramatically; second, we have gained new methods through which we can share discrete portions of our personal data, as well as comfort in doing so; and finally, both individuals and institutions are guiding their decisions based on multiple data sets—call it the Moneyball Approach to life.

Health:  There are two types of health-data-collecting devices, and it’s important to distinguish between the two.  The first are fitness-trackers, like FitBit and Fuel Band. While interesting and often fun, they are actually of less utility for health-tracking than the new breed of monitors that connect to smartphones and help people measure their temperature, blood pressure, weight, and other vital signs.

Though knowing when any individual is sick is not of interest to a community, much less to a government agency, when many people start getting sick at the same time, it can be of great interest.  First, because extra resources might need to flow into an area (think: more tamiflu), and second because special resources might be needed (think: shingles vaccine if an outbreak is detected).

The information that people would need to share is easily anonymized: initially, only 5-mile radius location and temperature.  That would be enough to alert public health officials of a nascent outbreak.  A Waze-like health app could then augment that rudimentary data with other important notifications.  Ultimately, a program like this could have huge benefits for the economy in terms of less money lost due to illness, and could potentially save lives as illnesses could be detected and treated much more quickly.

Financial:  One of the constant questions that countless magazines offer to answer is: am I getting the best deal on my mortgage/savings account/car insurance/credit cards.  And yet, people have a hard time talking about the terms of their mortgage, savings accounts, car insurance, and credit cards.  Further, few people know about all of the new financial tools springing up all the time.

Many people, for example, still turn to payday loans to cover for short-term debt.  Yet, those loans have an annual interest rate that can be anywhere from 400 percent to greater than 700 percent.  But developers could code an application that not only shared basic financial information, just location down to a 5-mile radius, and the terms and balances of various accounts—credit card, mortgage, checking, etc—but, further, coupled with information and forms from companies like ReadyForZero, which helps people plan to get out of debt, and Acorns, a micro-investing platform and Kiva, a micro-lending application.

Energy: One household turning up (or down) their thermostat doesn’t have a great impact on an electricity grid, but many households making changes in their thermostat can.  Aggregating energy use by community and then agreeing to try to keep to a certain level of consumption can increase the resiliency of an electricity grid, as well as keep more money in residents’ pockets.

Further, by having a baseline for comparison, individuals can see if they are falling above or below the average energy consumption for their home and can make changes as they see fit—using their own data which they do not share with any other organization, public or private.

Transportation:  Applications like Waze are only the beginning.  Devices like Automatic and a host of smartphone accessories can plug in to the computers in cars and record performance data.  Many states, such as Maryland, require emissions testing, but much of the pertinent information can either be recorded or inferred from the engine computer.  This is another area in which people could benefit from knowing how other people in similar situations are faring.  Drivers could optimize their cars, perhaps even going as far as updating the ECU—the engine computer—based on other drivers’ experiences and outcomes.

And then there is information about transportation infrastructure.  Already, applications like SeeClickFixFix311 and PublicStuff allow people to voluntarily send their city’s agencies requests for service.  Further, applications like Street Bump automatically send information about road conditions the same way that Waze does about traffic conditions

Further, sharing Waze information with municipalities can allow for on-the-go traffic management.  The pattern of traffic lights can be altered, public transit can respond to special events or abnormal conditions, and the flow of people from place to place can be eased to everyone’s benefit.  All that would need to be shared is specific location and ultimate destination (which, in the case of commuting, could be at a neighborhood or public transit stop-level location).

Sharing Economy: Companies like Park CircaAirBnB, and Relay Rides are the vanguard of what is often called “The Sharing Economy.”  Relay Rides sums up the benefit succinctly: “put your idle car to work.”  AirBnB puts people’s idle rooms or homes to work, and Park Circa puts people’s idle parking spaces to work.  The benefits for a community is a more efficient allocation of resources: people who don’t need to own a car (because they drive infrequently) can still have the benefit of a car, so they save money.  Conversely, people who own a car but use it only infrequently can still derive value from that car, value that they can then put back into the community.

But there’s a rub: the sharing economy is a largely unregulated one (though some see this as feature rather than bug), and the regime of data-collection in the regular (and regulated) economy does not extend into the sharing economy.  A shame, as there is much to gain for everyone if the sharing economy were regulated and its data mined.

The crux of the benefit goes to that idea of allocation of resources.  If a community has a host of rooms up on AirBnB, there might be less reason to build a new hotel; conversely if a few blocks sees a lot of AirBnB activity, a new restaurant might take the chance to open there.  Likewise, the use of Park Circa may indicate an inefficient allocation of parking resources, indicating the need either for a parking garage or different pricing for on-street parking in the area.  Relay Rides could provide data that local governments could use to plan and deploy programs for bike rentals stations, public transit, or other transportation efforts.

Small Data + Social = Big Data and All Its Benefits

The biggest benefit of Big Data is that it helps to ground decisions in reliable probabilities.  In the book Cognitive Surplus, author Clay Shirky tells the story of his small-town pizza shop, and how it sold pizza only by the whole pie.  When he visited New York City, he realized that pizza shops could sell by the slice, because there was a high enough probability that they would be able to sell the whole pie a slice at a time, given enough foot traffic.  His story proves the maxim that a large enough quantitative difference soon becomes a qualitative difference.

The same is true with the aggregation of small data.  The temperature readings coming out of a single household may not yield many insights, but the temperature readings coming out of thousands of households in a single city might.  The key to all of this is the social element: will we share certain data with one another the way we share links on Twitter, pictures on Instagram and Pinterest, and comments on Facebook and blogs?  If we do—providing, of course, that we get the privacy aspects right—we all stand to gain from making our small data big.

This post was originally published on the IBM Center for the Business of Government blog.

Recap: Hot Tech Trends in San Francisco for 2014

On Thursday February 20th, Tech in Motion: San Francisco rallied local techies. The gathering took place at Wish bar in the SoMa district of San Francisco. As drinks flowed, so did the conversations. San Francisco’s finest rolled into Wish Bar to discuss what tech trends they thought would hit big for 2014. For a Thursday evening, San Franciscan locals were more than happy to spend their free time learning a new name and adding a few more cards to their business card collection. Networking was in the air.


Tech in Motion conducted a survey to pin-point the leading topic of the night. There was no big surprise what they thought would hit the mainstream tech feed. The winner was Big Data – apache Hadoop! The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing. In a nutshell Apache Hadoop software is a framework that allows the distributed processing of large data sets across clusters of computers by using programming models that are simplified. More and more businesses are in need of a more advanced system to handle the immense amount of data that is collected – and big data paving the way for 2014.


Tech in Motion conducted a survey to pin-point the leading topic of the night. There was no big surprise what they thought would hit the mainstream tech feed. The winner was Big Data – apache Hadoop! The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing. In a nutshell Apache Hadoop software is a framework that allows the distributed processing of large data sets across clusters of computers by using programming models that are simplified. More and more businesses are in need of a more advanced system to handle the immense amount of data that is collected – and big data paving the way for 2014.

Another topic that scattered the powwows at the Tech in Motion mixer was high tech wearables. From coast to coast, techies are indulging themselves in wearable technology. Recently, New York Tech in Motion hosted an event for high tech fashion. The excitement of 3D printed shoes and LED clothing has definitely hit the west coast. Throughout the mixing and mingling, speculations on what hot new tech trends 2014 will offer permeated the room.IMG_3745

Tech in Motion members were also excited to discover Microsoft’s table, offering unlimited play time with Bing’s revived system, and a live engineer to answer any questions that arose. A buzz in the air of an exciting year to come has landed in San Francisco’s tech community.

Recap: San Francisco Startup to Success Panel

Tech in Motion: San Francisco kicked off 2014 with one of their biggest and most successful events to date. They hosted their first ever panel discussion ‘Startup to Success’ at the awesome StartupHouse in the hub of tech innovation in the city’s SoMa district.


StartupHouse offers an open plan co-working space for entrepreneurs and their teams. They have a cool, open space on the ground level where attendees enjoyed some refreshments, snacks, and networking before settling down to learn more about the star panel. StartupHouse was the perfect venue for a Tech in Motion meetup, especially with January’s startup theme and their involvement with incubators and accelerators bringing entrepreneurs to San Francisco.


In this stellar panel, not only did we welcome back some old favorites from meetups past, we also had some fresh new faces on our panel sharing stories about their startups. The amazing Perri Blake Gorman, CEO and Founder of Archive.ly moderated the panel and being the pro that she is, really delved into the interesting differences between each panelist from funding to hiring to going international with your company.


Jakob Storm, COO and Co-Founder of CrowdCuirty talked about how he and his team brought their idea from Denmark to the Bay Area. While it is a natural progression to move to the tech hub of the world, it was interesting to see how their process had differences having moved from Denmark to Buenos Aires to the Bay.

Scott Bedard, CTO and Co-Founder of thisMoment, offered a great depth of experience having been at Yahoo! and CNET previous to his most recent role. Perri talked to Scott about how he built his team to 15+ engineers and how this team evolved with the growth that thisMoment experienced.


We also heard from Kieran Farr, CEO and Co-Founder at Vidcaster, the market-leading video training and marketing platform with global clients such as VMware, MIT and Dell. Kieran’s backstory was really interesting as Vidcaster started out as a consumer platform. Having built their business idea upon this, they realized that if Vidcaster was going to be profitable they would have to focus on B2B customers.

Last but especially not least, Poornima Vijayashanker represented the ladies on our panel. Poornima is a role model for women in the often male dominated tech industry. She started out in the startup world as founding engineer at Mint.com, and from there founded Bizabee and Femgineer.  Femgineer provides education services to tech professionals and tech entrepreneurs. It was really interesting to listen to Poornima talk about funding and the difference between being a part of two companies – one VC funded and one bootstrapped and the challenges each can bring.

036 - Copy

The ‘Startup to Success’ Panel was definitely one of Tech in Motion San Francisco’s most interesting events. Thank you to our awesome members and we hope to see you all next month!

Recap: Results of Chicago’s Tech Trivia Event

On Wednesday, January 29th, Tech In Motion: Chicago hosted its first ever Tech Trivia event with over 150 techies present. As everyone arrived, teams were formed and then it was off to the races! We broke the event up into 4 rounds, worth 20 points each, plus a final bonus round. If you weren’t able to make it, here are the questions we asked – see how well you would have done!

Round 1: Logos

Answer Key - Logos_001

Round 2: General Chicago Trivia

  1. There were 7 Chicagoland companies that filed for an IPO in 2013. Name one. Textura, CDW, Claires, Potbelly, ASC Acquisition, Levy Acquisition, Hennessey Capital Acquisition
  2. In October 2013, John Tolva stepped down as the city’s CTO to join which environmentally focused engineering firm? PositivEnergy Practice
  3. Which online clothing seller did Groupon acquire in January of this year? Ideeli
  4. Howard Tullman, a veteran Chicago entrepreneur, was just named as the CEO of what company in November of 2013? 1871


Round 3: General Tech Trivia

  1. What does HTML stand for? Hyper Text Markup Language
  2. What programming language does Android use? Java
  3. What is the #1 blogging platform in the world? WordPress
  4. Larry Page and Sergey Brin founded what large internet based company? Google


Round 4: Multiple Answers

Name 10 of the 50 fastest growing companies by five-year revenue growth according to an article published by Crains in June of 2013.  Answers HERE


Bonus Round: Tech In Motion  Name 4 past Tech in Motion presenters.

And here are the results:


Congrats to our winners, Team V Nation, who walked away with $125 in Amazon gift cards and some great SWAG from our sponsors!


So how do you think you would have done? Any suggestions for us next time?

We want to thank everyone for coming out to the event and being such great participants. Thanks again to Jobspring Partners, Workbridge Associates, Microsoft, Start-Up Institute and Grind for helping make the event a huge success!

Recap: Toronto’s February Tech Mixer & Startup Demos

In The Burroughes’ swanky 6th floor event space, five startups local to the Toronto area came out to showcase their work, booth-style. The event kicked off with a quick introduction from Rebecca, the Toronto branch organizer, thanking Jobspring Partners for their continued financial backing, as well as giving a brief overview of the group to-date, which six events in now has over 900 members in Toronto, and garners over 200 RSVPs per event.

The introduction was followed by quick pitches from each of the presenting companies, who gave the crowd an idea of what their product does, and how the company was formed.

Top Hat, an education engagement platform, was the first to speak. Their mobile system is now deployed in over 330 universities worldwide.

cottage-event 083

Flixel was next up, a novel new app that allows users to take a “living photo”, where only parts of the photo move. The app was recently featured on America’s Next Top Model, where photos of models were created with moving clothing and hair. You can check out the project here.

cottage-event 050

AskForTask, a relatively new startup created by two university students, came out to showcase their website that allows people to post tasks that they need completed, and find people who may be skilled in that area, to complete them. The website currently has over 45,000 users and is expanding quickly.

cottage-event 045

Checkout 51 then spoke on their service, which is akin to reverse-couponing. The company sends a weekly email of local offers, and all the user needs to do is upload a receipt if they take part. When the account reaches $20 in rebates, a check is mailed to the user.

cottage-event 057

Last but definitely not least, Bitstrips came out to talk with the crowd about the mass success of their user-created cartoon app. Bitstrips is currently the most download app in the iPhone app store, and has only been released in the past year.

cottage-event 061

After the pitches, the companies set up demo tables where attendees were able to test out the programs, and talk with the founders and employees about what made their companies tick, how they got off the ground, and what they were looking forward to.

cottage-event 075

Web 3.0: The Next Big Thing is Here. How Will You Prepare?

Rey - Head ShotWe’ve been completely captured by Web 2.0 for the past decade. We upload videos to YouTube, tweet our thoughts, post about events, and take photos of our meals to share with the world – all in real time. This change to user-created content has driven a constant change in the way we communicate online. It’s changed our entire lives.

But what’s next? Is it possible that we are on the brink of the next evolutionary wave of peer-to-peer interaction?

Welcome to Web 3.0. The next big thing is here and it’s time you prepare.

Web 3.0 is the convergence of everything we have witnessed in the past up until today. The web will eventually become a single ecosystem where the worlds of social media and commerce collide. Consider the recent holiday season. We saw gift lists on Facebook, coupons on Pinterest, and millions of Christmas morning videos showing kids unwrapping the latest gadgets. We continue to showcase our latest purchases to our peers with validation in the form of “likes.”

The New Social Experience

Social Commerce is the next stop in online transactions. To give you a picture of what this looks like, imagine a guy tweeting his buddy a beer at the bar. In this scenario, the sender selects the account (credit, debit, checking, or savings) from which to push funds. When the transaction is successful, the receiver will chooses which of his accounts to drop the deposit into. While this scenario might seem a bit silly to some, there’s a larger picture to grasp.

Take this same technology and put it in the hands of celebrities like Paris Hilton, Katy Perry, or even non-profit organizations like the Red Cross. It’s easy to imagine the impact their Social Commerce campaigns can have when offering limited perfume, VIP concert tickets, and relief to those in need. With social platforms like YouTube, Vine, Instagram, and Facebook streaming video, the ability to sell at a much higher success rate is at our fingertips. The technology to convert the last frame in a streaming video to a checkout cart exists today. It’s only a matter of time before customers start purchasing the latest products and services at this level. The question is, what are you doing to act on that accessible technology?

It’s Time to Change the Game

One of the most important factors to consider is that the emergence of social payments will ultimately increase processing volume in the payments industry on a drastic level. This is why it is vital to pay attention and be part of the change. In the next few years, we can expect drastic change as smartphones, mobile wallets, and social networking are at the fingertips of every millennial.

We will continue to see change in the way customers make payments. Do you remember the days of going to a record store and using physical cash to make purchases? Do you remember the shift from this to online purchases with your credit card? This trend will continue as consumers use social media to make purchases and share their purchases. It is important to remember that all social media platforms have one thing in common – advertising as a source of revenue. We are not too far off from these platforms competing head-to-head with e-commerce giants like Amazon, eBay, or Google.

How You can Prepare

Are you utilizing a social media presence to increase sales? Are you offering exclusive deals to your followers on Twitter? Are you putting coupons on your Facebook page, encouraging fans to share and buy into your brand?

You should be.

Take a look at the big picture. Look at the future of social payments and where your company sits in the spectrum. Look at your current offerings and figure out a way to utilize social media into your campaign. For example, a local bakery could offer buy-one-get-one-free coupons that are exclusive to their Twitter or Facebook fans. You can even take it one step further and incorporate online videos that encourage consumers to share with their friends and family for a special deal. The return on this might end up being more than you bargained for.

It’s time to master the integration between commerce and social media. We can no longer be afraid of this emerging trend. By focusing on how we can better reach our target audience through social media TODAY, we will be better prepared for the explosive round of growth that takes place TOMORROW.

So the question is – How are you preparing?

Rey Pasinli, MBA, is Executive Director of Total-Apps and a featured sponsor of Tech in Motion.

Recap: #StaySecure in 2014!

Tech in Motion Boston kicked off the 2014 with a great demo and drinks event at the White Horse Tavern. Despite the bitter cold and the monstrosity that is the MBTA system, many Boston techies came out to network with their peers, gain advice, and hear helpful tips on how to stay secure for the New Year.

Lynnette Stracke, Channel Account Executive from our headlining sponsor, Sophos, kicked off the night with some insights into their award winning encryption, endpoint security, web, email, mobile and network security.


We had invited three great local security companies to come out, demo their products, and provide our techies with their top security tips for the New Year so as to not have a repeat of 2013: We’re looking at you NSA. 

Ken Smith, the Senior Security Solution Architect at Klogix, recommended to keep an eye on credit card statements, and make sure everything matches up to the purchases that your are making.

Michelle Drolet, CEO of Towerwall advised that passwords should be a minimum of 15 characters to which, a member of the audience shouted “Mine has 16!!” She also recommended getting phone antivirus, such as the Mobile Control system from Sophos, and warned to be cautious about how much information you are sharing with people across all social platforms.

Michelle also brought up the 4 E’s that her company follows in order to be successful.

  • Evaluate the risks
  • Establish a plan
  • Educate employees on security precautions
  • Enforce procedures and the need to always be on the lookout for new, potential risks

Even the CTO from Cloudlock, Ron Zalkind, took Michelle’s advice, referencing the 4 E’s and adding the importance of cleaning your phone. There’s a millions apps out there, get rid of the ones you never use! While he also warned to be cautious with your information, he felt that you should embrace the ideas of social media and cloud.


After these companies gave their thoughts and advice, our techies spent the rest of the night networking with one another and discussing the latest security trends and threats long after the event had ended.


As always, we would like to thank our two corporate sponsors Workbridge Associates and Jobspring Partners for being with us to lend their expert career advice.

Lastly, we would like to thank Sophos for being our headlining sponsor for this event! For more information about them and their services please visit their website.