Web, can be viewed as a CS attempt to share structureddata, and to integrate such data to build a Web-scalestructured KB. The World-Wide Web itself is perhaps thelargest CS system of all, encompassing everything wehave discussed.CHALLENGES AND SOLUTIONSHere, we discuss the key challenges of CS systems:How to recruit and retain users? Recruiting users is oneof the most important CS challenges, for which five majorsolutions exist. First, we can require users to make contributionsif we have the authority to do so (for example, amanager may require 100 employees to help build a company-widesystem). Second, we can pay users. MechanicalTurk for example provides a way to pay users on the Webto help with a task. Third, we can ask for volunteers. Thissolution is free and easy to execute, and hence is mostpopular. Most current CS systems on the Web (such asWikipedia, YouTube) use this solution. The downside ofvolunteering is that it is hard to predict how many userswe can recruit for a particular application.The fourth solution is to make users pay for service. Thebasic idea is to require the users of a system A to “pay” forusing A, by contributing to a CS system B. Consider forexample a blog website (that is, system A), where a user Ucan leave a comment only after solving a puzzle (called acaptcha) to prove that U is a human. As a part of the puzzle,we can ask U to retype a word that an OCR programhas failed to recognize (the “payment”), thereby contributingto a CS effort on digitizing written text (that is,system B). This is the key idea behind the reCAPTCHAproject. 34 The MOBS project 12,13 employs the same solution.In particular, it ran experiments where a user U canaccess a Web site (such as a class homepage) only afteranswering a relatively simple question (such as, is string“1960” in “born in 1960” a birth date?). MOBS leveragesthe answers to help build a data integration system. Thissolution works best when the “payment” is unintrusive orcognitively simple, to avoid deterring users from usingsystem A.The fifth solution is to piggyback on the user traces of awell-established system (such as building a spelling correctionsystem by exploiting user traces of a search engine,as discussed previously). This gives us a steadystream of users. But we must still solve the difficult challengeof determining how the traces can be exploited forour purpose.Once we have selected a recruitment strategy, we shouldconsider how to further encourage and retain users.Many encouragement and retention (E&R) schemes exist.We briefly discuss the most popular ones. First, we canprovide instant gratification, by immediately showing auser how his or her contribution makes a difference. 16Second, we can provide an enjoyable experience or a necessaryservice, such as game playing (while making a contribution).32 Third, we can provide ways to establish,measure, and show fame/trust/ reputation. 7,13,24,25 Fourth,we can set up competitions, such as showing top ratedusers. Finally, we can provide ownership situations, wherea user may feel he or she “owns” a part of the system, andthus is compelled to “cultivate” that part. For example,zillow.com displays houses and estimates their marketprices. It provides a way for a house owner to claim his orher house and provide the correct data (such as numberof bedrooms), which in turn helps improve the price estimation.These E&R schemes apply naturally to volunteering, butcan also work well for other recruitment solutions. Forexample, after requiring a set of users to contribute, wecan still provide instant gratification, enjoyable experience,fame management, and so on, to maximize user participation.Finally, we note that deployed CS systemsoften employ a mixture of recruitment methods (such asbootstrapping with “requirement” or “paying,” thenswitching to “volunteering” once the system is sufficiently“mature”).What contributions can users make? In many CS systemsthe kinds of contributions users can make are somewhatlimited. For example, to evaluate, users review, rate,or tag; to share, users add items to a central Web site; tonetwork, users link to other users; to find a missing boatin satellite images, users examine those images.In more complex CS systems, however, users often canmake a far wider range of contributions, from simple lowhangingfruit to cognitively complex ones. For example,when building a structured KB, users can add a URL, flagincorrect data, and supply attribute-value pairs (as lowhangingfruit). 3,5 But they can also supply inference rules,resolve controversial issues, and merge conflicting inputs(as cognitively complex contributions). 25 The challenge isto define this range of possible contributions (and designthe system such that it can gather a critical crowd of suchcontributions).Toward this goal, we should consider four important factors.First, how cognitively demanding are the contributions?A CS system often has a way to classify users intogroups, such as guests, regulars, editors, admins, and “dictators.”We should take care to design cognitively appropriatecontribution types for different user groups.38
Low-ranking users (such as guests, regulars) often wantto make only “easy” contributions (such as answering asimple question, editing one to two sentences, flagging anincorrect data piece). If the cognitive load is high, theymay be reluctant to participate. High-ranking users (suchas editors, admins) are more willing to make “hard” contributions(such as resolving controversial issues).Second, what should be the impact of a contribution? Wecan measure the potential impact by considering how thecontribution potentially affects the CS system. For example,editing a sentence in a Wikipedia page largely affectsonly that page, whereas revising an edit policy may potentiallyaffect million[s] of pages. As another example, whenbuilding a structured KB, flagging an incorrect data piecetypically has less potential impact than supplying an inferencerule, which may be used in many parts of the CSsystem. Quantifying the potential impact of a contributiontype in a complex CS system may be difficult. 12,13 Butit is important to do so, because we typically have farfewer high-ranking users such as editors and admins(than regulars, say). To maximize the total contribution ofthese few users, we should ask them to make potentiallyhigh-impact contributions whenever possible.Third, what about machine contributions? If a CS systememploys an algorithm for a task, then we want humanusers to make contributions that are easy for humans, butdifficult for machines. For example, examining textualand image descriptions to decide if two products match isrelatively easy for humans but very difficult for machines.In short, the CS work should be distributed betweenhuman users and machines according to what each ofthem is best at, in a complementary and synergistic fashion.Finally, the user interface should make it easy for users tocontribute. This is highly non-trivial. For example, howcan users easily enter domain knowledge such as “no currentliving person was born before 1850” (which can beused in a KB to detect, say, incorrect birth dates)? A naturallanguage format (such as in openmind. org) is easy forusers, but difficult for machines to understand and use,and a formal language format has the reverse problem. Asanother example, when building a structured KB, contributingattribute-value pairs is relatively easy (asWikipedia infoboxes and Freebase demonstrate). Butcontributing more complex structured data pieces can bequite difficult for naive users, as this often requires themto learn the KB schema, among others. 5How to combine user contributions? Many CS systemsdo not combine contributions, or do so in a loose fashion.For example, current evaluation systems do not combinereviews, and combine numeric ratings using relativelysimple formulas. Networking systems simply link contributions(homepages and friendships) to form a social networkgraph. More complex CS systems, however, such asthose that build software, KBs, systems, and games, combinecontributions more tightly. Exactly how this happensis application dependent. Wikipedia, for example, letsusers manually merge edits, while ESP does so automatically,by waiting until two users agree on a common word.No matter how contributions are combined, a key problemis to decide what to do if users differ, such as whenthree users assert “A” and two users “not A.” Both automaticand manual solutions have been developed for thisproblem. Current automatic solutions typically combinecontributions weighted by some user scores. The work 12,13for example lets users vote on the correctness of systemcomponents (the semantic mappings of a data integrationsystems in this case 20 ), then combines the votes weightedby the trustworthiness of each user. The work 25 lets userscontribute structured KB fragments, then combines theminto a coherent probabilistic KB by computing the probabilitiesthat each user is correct, then weighting contributedfragments by these probabilities.Manual dispute management solutions typically let usersfight and settle among themselves. Unresolved issuesthen percolate up the user hierarchy. Systems such asWikipedia and Linux employ such methods. Automaticsolutions are more efficient. But they work only for relativelysimple forms of contributions (such as voting), orforms that are complex but amenable to algorithmic manipulation(such as structured KB fragments). Manual solutionsare still the currently preferred way to combine“messy” conflicting contributions.To further complicate the matter, sometimes not justhuman users, but machines also make contributions.Combining such contributions is difficult. To see why,suppose we employ a machine M to help createWikipedia infoboxes. 35 Suppose on Day 1 M asserts population= 5500 in a city infobox. On Day 2, a user U maycorrect this into population = 7500, based on his or herknowledge. On Day 3, however, M may have managed toprocess more Web data, and obtained higher confidencethat population = 5500 is indeed correct. Should M overrideU’s assertion? And if so, how can M explain its reasoningto U? The main problem here is it is difficult for amachine to enter into a manual dispute with a humanuser. The currently preferred method is for M to alert U,and then leave it up to U to decide what to do. But thismethod clearly will not scale with the number of conflictingcontributions.39
- Page 1 and 2: VOLUME FOUR STRATEGIC PERSPECTIVESL
- Page 3 and 4: VOLUME FOUR STRATEGIC PERSPECTIVESL
- Page 5 and 6: VOLUME FOUR STRATEGIC PERSPECTIVESL
- Page 7: VOLUME FOUR STRATEGIC PERSPECTIVESL
- Page 10 and 11: 12CHAPTER 12INTRODUCTION TO STRATEG
- Page 12 and 13: 12.1 Strategic Leadership: Defining
- Page 14 and 15: mandates or resolutions that would
- Page 16 and 17: and ambiguity, aspiring strategic l
- Page 18 and 19: 12.2 National Security StrategyThe
- Page 20 and 21: within our borders has always been
- Page 22 and 23: front common challenges like violen
- Page 24 and 25: and our strategy, not sector earmar
- Page 26 and 27: thinking about organizations. She m
- Page 28 and 29: A systemic approach to failure is m
- Page 30 and 31: The late W. T. Grant Company is a r
- Page 32 and 33: the resources - setting the directi
- Page 34 and 35: focal point for describing and inte
- Page 36 and 37: Consequently, we do not restrict th
- Page 38 and 39: paragraphs, Web pages, then edit an
- Page 42 and 43: How to evaluate users and contribut
- Page 44 and 45: 13CHAPTER 13LEADING PUBLIC &VOLUNTE
- Page 46 and 47: 13.1 Leadership for Volunteers:The
- Page 48 and 49: 13.2 Take Root: Volunteer Managemen
- Page 50 and 51: QualificationsClearly list educatio
- Page 52 and 53: and effectively track their volunte
- Page 54 and 55: • Understand rules for recognitio
- Page 56 and 57: • Send a birthday card.• Submit
- Page 58 and 59: and tested more than six decades af
- Page 60 and 61: specific interests of the donors, v
- Page 62 and 63: tain) tax-exempt status from the In
- Page 64 and 65: Smucker, 1999).The Internal Revenue
- Page 66 and 67: culture is necessary to ensure the
- Page 68 and 69: 13.4 The New Look of TransparencyBy
- Page 70 and 71: ees for a couple of reasons: One, i
- Page 72 and 73: 13.5 Public and Private Management:
- Page 74 and 75: TABLE 1:FUNCTIONS OF GENERAL MANAGE
- Page 76 and 77: 3. Career System. The model corpora
- Page 78 and 79: islative charter - the Clean Air Ac
- Page 80 and 81: In controlling performance, Chapin
- Page 82 and 83: 14CHAPTER 14AIRPOWER ASSTRATEGIC LA
- Page 84 and 85: 14.1 Strategic Air Power: Fulfillme
- Page 86 and 87: carry it out. Their daylight raids
- Page 88 and 89: you did not rely on strategic bombi
- Page 90 and 91:
14.2 Warden and the Air Corps Tacti
- Page 92 and 93:
ecomes one of applying sufficient i
- Page 94 and 95:
tification, and a Jominian claim to
- Page 96 and 97:
courage the rapid and widespread ex
- Page 98 and 99:
it to influence physical players in
- Page 100 and 101:
14.4 Basic Air Force DoctrineAF Doc
- Page 102 and 103:
earthquake-stricken Haiti. The worl
- Page 104 and 105:
perspective. Airmen do not divide u
- Page 106 and 107:
Command and ControlCommand and cont
- Page 108 and 109:
14.5 Should the US Maintain the Nuc
- Page 110 and 111:
form of human government.” 20 Dem
- Page 112 and 113:
obtainable goal. See the Global Zer
- Page 114 and 115:
15CHAPTER 15ORGANIZATIONAL CULTURE
- Page 116 and 117:
15.1 Organizational CultureBy Doria
- Page 118 and 119:
Review. This action strives to unco
- Page 120 and 121:
gram will serve and then having the
- Page 122 and 123:
ticipating the changes being made b
- Page 124 and 125:
Many years of working with change p
- Page 126 and 127:
At the least, the areas of concern
- Page 128 and 129:
15.4 Developing an Innovative Cultu
- Page 130 and 131:
CONCLUDING THOUGHTSIn an ever-chang
- Page 132 and 133:
global issues. Businesses that poss
- Page 134 and 135:
— Sees the big picture—the shif
- Page 136 and 137:
16CHAPTER 16STRATEGIC COMMUNICATION
- Page 138 and 139:
16.1 Principles of Strategic Commun
- Page 140 and 141:
16.2 The Art of NegotiationBy Brend
- Page 142 and 143:
16.3 Negotiating Effectively Across
- Page 144 and 145:
hidden areas can act as cultural ho
- Page 146 and 147:
and four conflict styles. Hammer be
- Page 148 and 149:
maintaining the relationship. As th
- Page 150 and 151:
25 Mitchell R. Hammer, “Chapter 1
- Page 152 and 153:
tural, socioeconomic, and psycholog
- Page 154 and 155:
for the win-win," during which time
- Page 156 and 157:
interests and, at worst, as a gun s
- Page 158 and 159:
Public diplomacy is surely about mu
- Page 160 and 161:
But public diplomats do not have th
- Page 162 and 163:
Photo courtesy of the familyThe LEA
- Page 164:
THE CADET OATHI pledge that I will