Inside view on ads review

Posted by David W. Baker, Director of Engineering, Advertising

Cross-posted from the Official Google Blog.

This is the first in a series of posts that will provide greater transparency about how we make our ads safer by detecting and removing scam ads. -Ed.

A few weeks ago, we posted here about our efforts in fighting bad ads, and we shared a video with the basics of how we do it. Today I wanted to delve a little deeper and give some insight into the systems we use to help prevent bad ads from showing. Our ads policies are designed with safety and trust in mind—we don’t allow ads for malicious downloads, counterfeit goods, or ads with unclear billing practices, to name a few examples. In order to help prevent these kinds of ads from showing, we use a combination of automated systems and human input to review the billions of ads submitted to Google each year. I’m one of many engineers whose job is to help make sure that Google doesn’t show bad ads to users.

We’ve designed our approach based on a three-pronged strategy, each focused on a different dimension of the problem: ads, sites, and advertiser accounts. These systems are complementary, sharing signals among each other so that we can comprehensively attack bad ads.

For example, in the case of a site that is selling counterfeit goods, this three-pronged approach aims to look for patterns that would flag such a site and help prevent ads from showing. Ad review notices patterns in the ads and keywords selected by the advertiser. Site review analyzes the entire site to determine if it is selling counterfeit goods. Account review aims to determine if a new advertiser is truly new, or is simply a repeat offender trying to abuse Google’s advertising system. Here’s more detail on how we review each of these three components.

Ad Review
An ad is the snippet of information presented to a user, along with a link to a specific webpage, or landing page. The ads review system inspects individual ads and landing pages, and is probably the system most familiar to advertisers. When an advertiser submits an ad, our system immediately performs a preliminary examination. If there’s nothing in the ad that flags a need for further review, we tell the advertiser the ad is “Eligible” and show the ad only on google.com to users who have SafeSearch turned off. If the ad is flagged for further review, in most cases we refer to the ad as “Under Review” and don’t show the ad at all. From there, the ad enters our automated pipeline, where we employ machine learning models, a rules engine and landing page analysis to perform a more extensive examination. If our automated system determines an outcome with a high degree of confidence, we will either approve the ad to run on Google and all of our partners (“Approved”), approve the ad to show for appropriate users in specific locations (“Approved - Limited”) or reject the ad (“Disapproved”). If our automated system isn’t able to determine the outcome, we send the ad to a real person to make a final decision.

Site Review
A site has many different pages, each of which could be pointed to by different ads, often known as a domain. Our site review system identifies policy issues which apply to the whole site. It aggregates sites across all ads from all advertisers and regularly crawls them, building a repository of information that’s constantly improving as new scams and new sites are examined. We store the content of advertised sites and use both machine learning models and a rules engine to analyze the sites. The magic of the site review system is it understands the structure of language on webpages in order to classify the content of sites. Site review will determine whether or not an entire site should be disabled, which would prevent any ads leading to that site showing from any account. When the automated system isn’t able to determine the outcome with a high degree of confidence, we send it to a real person to make a decision. When a site is disabled, we tell the advertiser that it’s in violation of “Site Policy.”

Account Review
An account is one particular advertiser’s collection of ads, plus the advertiser’s selections for targeting and bidding on those ads. An account may have many ads which may point to several different sites, for example. The account review system constantly evaluates individual advertiser accounts to determine if the whole account should be inspected and shut down for policy violations. This system “listens” to a variety of signals, such as ads and keywords submitted by the advertiser, budget changes, the advertiser’s address and phone number, the advertiser’s IP address, disabled sites connected to this account, and disapproved ads. The system constantly re-evaluates all accounts, incorporating new data. For example, if an advertiser logs in from a new IP address, the account is re-evaluated to determine if that new signal suggests we should take a closer look at the content of the advertiser’s account. If the account review system determines that there is something suspect about a particular account with a high degree of confidence, it automatically suspends the account. If the system isn’t sure, it stops the account from showing any ads at all and asks a real person to decide if the account should be suspended.

Even with all these systems and people working to stop bad ads, there still can be times when an ad slips through that we don’t want. There are many malicious players who are very persistent—they seek to abuse Google’s advertising system in order to take advantage of our users. When we shut down a thousand accounts, they create two thousand more using different patterns. It’s a never-ending game of cat and mouse.

We’ve put a great deal of effort and expense into building these systems because Google’s long-term success is based on the trust of people who use our products. I’ve focused my time and energy in this area for many years. I find it inspiring to fight the good fight, to focus on the user, and do everything we can to help prevent bad ads from running. I’ll continue to post here from time to time with additional thoughts and greater information about how we make ads safer by detecting and removing scam ads.

Announcing the 2012 Google Policy Fellows

Posted by Pablo Chavez, Director of Public Policy

We’re excited to announce the 2012 class of Google Policy Fellows, and we’re expecting great things from the 15 students selected for the fifth summer of the Google Policy Fellowship. Our host organizations selected the 2012 fellows from over 1300 impressive submissions. The 2012 class includes undergraduate and graduate students from 12 schools, studying history, public policy, economics, science and technology, computer science, engineering and law.

Congratulations to the 2012 Google Policy Fellows!

Derek Attig, University of Illinois, ALA Washington
Justin Kaufman, George Washington University, Public Knowledge
Lassana Magassa, University of Washington, New America Foundation
Daniel Lieberman, George Washington University, Future of Music Coalition
Anjney Midha, Stanford University, Technology Policy Institute
Yana Welinder, Harvard University, Electronic Frontier Foundation
Jonathan Miller, Georgetown University, Center for Democracy & Technology
Michael Corliss, University of Illinois, TechFreedom
Kieran Bergmann, University of Ottawa, Citizen Lab
Sumitra Nair, Virginia Polytechnic Institute and State University, Joint Center for Political and Economic Studies
Andrew Blanco, Stanford University, Creative Commons
Brenda Villanueva, University of Maryland, National Hispanic Media Coalition
Brian Picone, Brown University, Competitive Enterprise Institute
Laurie Birbilas, McGill University, Canadian Internet Policy and Public Interest Clinic
Leonard Hyman, University of Southern California, Internet Education Foundation

The 2012 Fellows will spend 10 weeks this summer at our host organizations working on Internet and technology policy issues including free expression, privacy, security, and intellectual property.

Thank you to everyone who applied. Please sign up here to follow our program announcements, and visit google.com/policyfellowship for more information.

Checking in with the Global Network Initiative

Posted by Bob Boorstin, Director, Public Policy and Lewis Segall, Senior Ethics and Compliance Counsel

No matter who or what you are, opening up to outside scrutiny isn’t an easy or comfortable process. But that's what we agreed to do a few years ago when we helped found the Global Network Initiative (GNI), an amalgam of companies, human rights activists, socially responsible investors and academics formed in response to actions by governments that endanger free expression on the global Internet.

The objectives of GNI are both simple and incredibly complex: promote and support free expression and privacy online; subscribe to principles and follow guidelines supported by measures of transparency and accountability; and educate people and engage policymakers around the world in an effort to create a more open and free Internet.

In starting GNI, the founding companies — Google, Microsoft and Yahoo — agreed to bring in outside assessors to review how we were doing against GNI principles. Our agreement to conduct these assessments is an important part of the organization's credibility.

Now these first assessments are finished and the results have been released as part of GNI's annual report released yesterday. After reviewing them, the non-company members of GNI have told us that while we're by no means perfect, the assessments are credible and rigorous and demonstrate that companies are making progress — a concrete step in our efforts to build trust not only with our GNI partners but with all our users.

The activities of Google to promote free expression and privacy around the world extend well beyond GNI. However, being a part of this group is a compelling opportunity, since it brings together diverse stakeholders and provides a unique forum to address the risks to a free and open internet. Along with the GNI, we welcome other companies and groups to join this effort.

Bringing the Halls of Museums Into Classrooms Everywhere

Posted by Piotr Adamczyk, Google Art Project Team

The last known photo portrait of Abraham Lincoln. A masterpiece made up of thousands of tiny painted dots. Scenes of the American West that will leave you breathless. Some of our nation’s greatest treasures -- all at your fingertips.

In a few hours I will join art enthusiasts, students, and local leaders at the Art Institute of Chicago to announce a major expansion of the Google Art Project. We are thrilled to have partnered with 151 museums in 40 countries including 29 American institutions in 16 cities.

Now anyone with an Internet connection can take a virtual field trip to see some of the finest art around the world and right here in America. No permission slip, sack lunch or bus fare needed! All before lunch, from their desk, a student could walk through the East Room of the White House, visit one of the greatest collections of the American West in the world at the Denver Art Museum, and take a quick trip to Massachusetts to see some of the great illustrations at the Norman Rockwell Museum. They could, of course, also go see South African rock art, street graffiti from Brazil or Australian aboriginal art.

All told, the Art Project puts over 30,000 works of art at your fingertips and allows you to:

• Explore inside museums: We’ve adapted mapping technology to take 360-degree images of the interior of selected galleries which were then stitched together, enabling smooth navigation of more than 385 rooms within the museums. You can also explore the gallery interiors directly from within Street View in Google Maps.
• Discover history: Our partners have provided the backstory of many of the works in the Art Project and the artists that created these pieces. The Art Project is a fantastic classroom tool with hundreds of short educational videos created by SmartHistory presented by Khan Academy.
• Get up close: We now have 46 artworks available with our “gigapixel” photo capturing technology, photographed in extraordinary detail using super high resolution so you can study details of the brushwork and patina that would be impossible to see with the naked eye.
• Curate your own collection - build a personalized collection, provide your own commentary, and share it with your friends and family.

To learn even more about the Google Art Project and today’s announcement read this post on the Official Google Blog.