There is no excuse for the abuses we are learning about at GSA. We should all be bothered by the waste, fraud and abuse and I hope the government moves fast to correct the failures of leadership evident in these problems.

Something else should bother us. The bad apples at GSA who either conducted these crimes or allowed them to happen are reflecting poorly on those at GSA who are doing great work. I hate that. There are some great professionals at GSA who work hard to do the right thing every day, including some world class best technologists that could probably use some love right now.

For all the good people at GSA, especially the techies, I wish you well and thank you for your continued service.

And for you readers who are perhaps not so familiar with the great IT work being done at GSA, let me mention one example, USASearch.gov. This is a perfect exemplar of one of our government’s great IT projects. It helps the government get the right information to citizens and does so in an incredibly economical way. And it does so in a way where common things are provided to all government agencies efficiently. That is exactly the sort of thing the GSA is supposed to be doing. Because of USASearch.gov, any agency in government that needs a solution to deliver better results to citizens through search can leverage a well thought out, proven, tested solution.

As you may recall, in 2011 GSA was the winner of the Government Big Data Solutions Awardwith that capability. The below is from the post we did on that award:

GSA USASearch Wins 2011 Government Big Data Solutions Award

 

The Government Big Data Solutions Award was established to highlight innovative solutions and facilitate the exchange of best practices, lessons learned and creative ideas for addressing Big Data challenges. The top five nominees and overall winner was announced at Hadoop World in New York City on November 8 2011.

The Government Big Data Solutions Award Program is coordinated by CTOLabs.com.  The 2011 judging panel included: Doug Cutting, creator of Apache Hadoop and architect at Cloudera, Alan Wade, former CIA and IC CIO, Ryan LaSalle, Director of Accenture Cyber R & D, Ed Granstedt, Senior VP Director of the QinetiQ Strategic Solutions Center and Chris Dorobek, founder, editor and publisher of DorobekInsider.com.

The top five honorees of the Government Big Data Solutions Award are:

• USA Search: Hosted search services over more than 500 government sites.  Provides search and suggestion services plus analytical tool dashboards.
• GCE Federal: Cloud-based financial management solutions.
• PNNL Bioinformatics: Advancing understanding of health, biology, genetics and computing.
• SherpaSurfing: A cybersecurity solution that analyzes trends, finds malware, and writes alerts.
• US Department of State, Bureau of Consular Affairs: Large data set with critically important applications for citizen service and national security.

The winner of the 2011 Government Big Data Solutions Award is the USA Search Program of the US General Services Administration Office of Citizen Services and Innovative Technologies.

Award judges saw the USASearch Program as a great example of solving Big Data problems to improve government agility and provide better service for less. In line with the GSA’s cost-saving “build once, use many times” paradigm, USASearch has provided hosted search services for USA.gov and, through its Affiliate Program, over 500 other government websites. This is done in an incredibly cost-effective way, especially for the agencies involved (which receive these services from GSA for free).

In 2010, USASearch adopted an open architectural model to better exploit shared solutions and open source technology. This model leverages Cloudera’s Distribution including Apache Hadoop (CDH3). The move brought further cost savings and scalable shared search services, which drove up usage. USASearch adopted Hadoop to improve the search results by aggregating and analyzing  information on what users are searching, their success in finding it, the time of the search, the affiliate, the results, and which results users click on, among other trends. To do so across hundreds of affiliates with growing traffic, USASearch considered scaling up or dividing its database systems, but they knew that these solutions would be costly and temporary.  Instead, it turned to  HDFS, Hadoop, and Apache Hive—a big data system that could grow cost effectively and without downtime, be naturally resilient to failures, and sensibly handle backups.

The overhaul of USASearch’s analytics is a dramatic success story. In the space of a few months, USASearch went from having a brittle and hard-to-scale RDBMS-based analytics platform to an agile, scalable Hadoop-based system.  By using a state-of-the-art open source technology, USASearch has created a radically different search service that transforms the customer experience Hadoop’s uses and effects will continue to expand as more data sources and tools are added. Having a government-owned and controlled search service allows provides insight on the needs and concerns of Americans to drive enhancements to other delivery channels. The public has a much improved experience when interacting with the government due to USASearch.

The GSA is to be congratulated for their mission-focused, citizen-centered, open approach to a big data challenge and a resulting solution that improves the experience of a broad swath of users of federal services. On behalf of our judges and the many citizens who use this capability on a daily basis we say thank you, and congratulations on this well deserved recognition.

Related articles

• The Best Way To Get Hadoop Is Getting Better: CDH 4 in Final Beta (ctovision.com)
• Datameer provides end-user focused BI solutions for big data analytics (ctovision.com)
• The performance of the GSA is Congress’ responsibility (thehill.com)