skip to primary navigationskip to content

Latest News

Tracking inequality in India: the story of a pioneer

By amb206 from University of Cambridge - Big data. Published on Jul 04, 2017.

The widening gap between India’s rich and poor is captured by the National Sample Survey (NSS), an organisation founded in 1950, which gathers data from roughly 14,000 Indian villages and localities to provide a snapshot of how the population at large is faring. The NSS and its pioneering role in the measurement of poverty and inequality are some of the important subjects addressed by a conference that starts tomorrow (5 July 2017) in Cambridge to explore how different modern societies have measured social and economic disparity.

Since Indian Independence in 1947, the NSS has conducted more than 70 rounds of surveys, providing much-needed data about household consumption, social inequality, educational attainment and healthcare outcomes. NSS data serves as a backbone to Indian economic planning, public welfare provision and academic research.

The story behind the NSS goes back to 1913, when a brilliant young man called Prasanta Chandra Mahalanobis arrived at King’s College, Cambridge, to study mathematics.

It is said that Mahalanobis had intended to become a student in London but applied to King’s after visiting its world-famous chapel and missing the last train back to the capital. He graduated with a BA in natural science, receiving top marks in his physics final exam.

During his time at Cambridge, he interacted with another outstanding Indian mathematician, Srinivasa Ramanujan. Influenced by the British journal Biometrika, Mahalanobis began experimenting with new statistical methods for studying and measuring large-scale phenomena – occurrences so widespread and diverse by nature that they are difficult to gauge.

A man of diverse scientific interests, Mahalanobis combined statistics with other emerging disciplines, including anthropology, physics and economics, to develop novel approaches for estimating population distribution, crop yields and household consumption.

Mahalanobis is known for his pioneering work in descriptive statistics – and his name is remembered by the ‘Mahalanobis distance’, a measurement used in studies of population. For many years he taught at Presidency College (Kolkata) where, in 1931, he was responsible for founding the Indian Statistical Institute (ISI).

Today the ISI employs a staff of more than 1,000 people and is a leading international centre for research in applied mathematics, data science and computing.

With funding from the Philomathia Foundation, Dr Poornima Paidipaty (Faculty of History) has embarked on a study focusing on Mahalanobis’s most important contribution at the ISI: his visionary work on the development of large-scale surveys of India’s rural population in response to the country’s drive to realign itself as an industrial force with global reach.

Her research is part of a much larger project on ‘Historicising the Measurement of Inequality’, which is directed by Dr Pedro Ramos-Pinto and examines global histories of quantifying and framing socio-economic disparity.

Starting in the late 1930s, the ISI undertook a series of pioneering pilot surveys to gauge Indian household incomes at a time of huge social and historical upheaval. Sampling offered Indian scientists new tools for generating data on phenomena that had never been comprehensively or accurately measured before. In its early years as a research and training centre, the ISI used sampling to study everything from changing patterns in tea consumption to estimating crop acreage.

This research became more urgent after Independence, when government planners needed more reliable economic data to frame programmes aimed at rapid industrialisation, poverty alleviation and development. Lacking a strong household income tax regime, Indian bureaucrats lacked the fine-grained statistical information used by economists in developed countries to accurately estimate GDP.

Mahalanobis and his colleagues at the ISI offered a unique solution to these problems and designed a pioneering large scale sampling exercise to estimate the size, composition and condition of the Indian economy. As an approach to measurement, it was an original (and at the time, highly risky) endeavour. Many doubted that random sampling could accurately represent the totality of Indian social and economic life.

In 1950, Mahalanobis launched the National Sample Survey (NSS) to undertake the ambitious task of providing a comprehensive picture of India’s domestic economy. In first rounds of research, 1833 villages and residential areas were surveyed. This limited sample was used to represent the nation as a whole, which totalled roughly 360 million people at the time.

During this early period, critics complained that urban areas were over-represented and that surveyors were unfamiliar with the struggles and transformations facing remote regions and rural villages. It took many years for Mahalanobis and colleagues to design a survey that would capture, with an acceptable level of accuracy, the data that the government sought.

Due in part to his widespread academic interests, and his interactions with intellectuals from fields other than mathematics, Mahalanobis’s work incorporated cutting edge research in the social and computational sciences of the postwar era. He collaborated with top economists and mathematicians from around the world, and brought leading scientists to Kolkata for extended periods of time.

Ronald Fisher, JBS Haldane, Norbert Wiener, Andrey Kolmogorov, Jerzy Neyman, Joan Robinson and Simon Kuznets were among the many researchers sponsored by the ISI to collaborate on the Institute’s teaching and ongoing survey efforts in the 1950s and 1960s.

During its first decade, NSS researchers had to address numerous and complicated issues. What size and distribution of survey sites would best represent the nation in its entirety? How should surveyors account for India’s significant informal sector and for labour that was paid in kind, rather than cash?

Measuring national productivity required that researchers account for all productive labour – not just monetised transactions. Similarly, how should surveyors include women’s labour? Survey teams had to build rapport with their subjects, and in many cases, even teach them how to estimate monthly consumption and expenditure. The accuracy of data relied on social ties and mutual education – not just rote completion of questionnaires.

Over time, the NSS not only became a valued and relied upon institution, it influenced researchers and policymakers around the globe. Chinese officials sent their statisticians to Kolkata to learn from Mahalanobis’s staff in the 1950s, and the ISI served as a model for the American statistician Gertrude Cox, for the organisation of statistical training in the USA.

With her background in science studies and South Asian history, Paidipaty is well-equipped to understand the technical as well as the social relationships that allowed Indian planners and scientists to define and steer the national economy. Her research draws on the extensive archives of the ISI, which offer unique insights as to how Indian household life was measured in the early decades after Independence and Partition, and how policymakers framed and understand shifting standards of living.

Paidipaty’s work demonstrates that sampling, as a technique of economic measurement, was intimately tied to mid-century economic planning. Under Nehru’s leadership, the Indian state focused its developmental efforts on rapid industrialisation and growth, but achieving these objectives required new tools for defining and measuring the national economy. What were the different, discreet parts of an economy and how did they relate to one another?

Pinning down such abstractions, and offering concrete, tangible data, was indispensible to the work of managing India’s planned economy. The early history of sampling roughly overlapped with early experiments in economic planning. Mahalanobis was a member of India’s Planning Commission from 1953 until 1967, and directed the nation’s Second Five Year Plan.

In 2014, India’s government dissolved the Planning Commission, arguing that pro-growth policies ought to be achieved through unfettered markets rather than planned policy interventions. Yet, even without a formal planning apparatus, the significance of large-scale sampling has only grown over the last 70 years.

Since the 1980s, economists around the world, including those at the World Bank and the IMF, have embraced and underscored the importance household sampling. Not only do they provide large-scale aggregative statistics, they are a crucial source of fine-grained and qualitatively rich data.

The NSS has been an on-going subject of debate amongst economists, but is also a crucial source of information. Angus Deaton, the recipient of the 2015 Nobel prize in economics, in some of his most influential work used NSS data to help the Indian government recalibrate how it defined and measured poverty. Within the current Indian context, in which economic growth and rising inequality are once again at the centre of public debate, it has become all the more important to understand the history of data, how it is produced and what numbers really represent.

As a nation, India is undergoing profound transformation, but rapid growth has come hand in hand with rising inequality as well as growing disparity between rural and urban areas. NSS data remains one of the best resources for understanding and tracking these changes. As more of this information circulates in the public domain, it becomes all the more crucial to appreciate how such data is produced. Paidipaty’s work on the history of the NSS offers a fascinating glimpse into one of the most significant and early mid-century precursors to contemporary developments in big data.

'Measuring Matters: Histories of Assessing Inequality' takes place from 5 to 7 July 2017 at the Alison Richard Building, 7 West Road, Cambridge. The conference is sponsored by Cambridge's Centre for Research in the Arts, Social Sciences and Humanities (CRASSH). 

India’s booming business centres and gleaming shopping malls mask a grimmer reality. While one section of the population gets richer, another section gets poorer. In the countryside, farmers and others ‘left behind’ by the economic surge find themselves in increasingly desperate circumstances. In many cases their plight, exacerbated by crippling debt, has led to suicide.

Within the current Indian context, in which economic growth and rising inequality are once again at the centre of public debate, it has become all the more important to understand the history of data, how it is produced and what numbers really represent.
Women working in the rice paddy fields in Odisha, one of the the poorest regions of India

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Inaugural $100,000 Nine Dots Prize winner chosen from more than 700 worldwide entries

By sjr81 from University of Cambridge - digital society. Published on May 31, 2017.

Up against competition from over 700 other entrants from around the world, Williams’ 3,000-word answer to the set question ‘Are digital technologies making politics impossible?’ was deemed the most original and innovative by the ten-strong Board of leading academics, journalists and thinkers.

His entry Stand Out of Our Light: Freedom and Persuasion in the Attention Economy argues that digital technologies are making all forms of politics worth having impossible as they privilege our impulses over our intentions and are ‘designed to exploit our psychological vulnerabilities in order to direct us toward goals that may or may not align with our own’. He covers:

  • How the ‘distractions’ produced by digital technologies are much more profound than minor ‘annoyances’
  • How so-called ‘persuasive’ design is undermining the human will and ‘militating against the possibility of all forms of self-determination’
  • How beginning to ‘assert and defend our freedom of attention’ is an urgent moral and political task

As well as the US$100,000 prize money, Williams has been awarded a book deal with Cambridge University Press for a book in which he will develop his ideas on this topic. He will be supported by the editorial team at Cambridge University Press and will spend a term at the Centre for Research in the Arts, Social Sciences and Humanities (CRASSH), Cambridge University.

Born in Cape Canaveral, Florida and raised in Texas, Williams is currently a doctoral candidate at the Oxford Internet Institute and Balliol College, Oxford, where he researches the philosophy and ethics of attention and persuasion as they relate to technology design. He is also a member of the Digital Ethics Lab at Oxford and a visiting researcher at the Uehiro Centre for Practical Ethics. Prior to that he worked for over ten years at Google, where he received the Founders’ Award – the company’s highest honour – for his work on advertising products and tools. He is also a co-founder of the Time Well Spent campaign, a project that aims to steer technology design towards having greater respect for users’ attention. He holds a master’s in design engineering from the University of Washington and as an undergraduate studied literature at Seattle Pacific University.

On being awarded the Prize, Williams said: “I'm honoured, grateful, heartened, energised and overjoyed to have won this opportunity. I know that many others thought deeply about this question and put substantial time, attention and care into answering it. I'm looking forward to getting to work on producing a book that is worthy of the competition.

“As Neil Postman pointed out in the 1980s, we’re far more attuned to Orwellian threats to freedom such as coercion and force, than to the subtler, more indirect threats of persuasion or manipulation of the sort Aldous Huxley warned us about when he predicted that it’s not what we fear but what we desire that will control us. Yet today these Huxleyan threats pose the far greater risk, and I’m extremely encouraged that the Nine Dots Prize Board has chosen to give its attention to these pressing matters. Their important question is not only compelling but also timely, and this competition is a fascinating and original way of putting such a crucial subject on the societal radar.”

The Nine Dots Prize was established to promote and encourage innovative thinking to address problems facing the modern world. It is judged anonymously by a Board chaired by Professor Simon Goldhill, Director of CRASSH.

Professor Goldhill said: “This competition was uniquely exciting: all the entries were anonymous, and we had no idea whether we were reading the proposal of a professor, a novelist, a postman, a student or a lawyer. It turned out afterwards we had plenty of all of these among our more than 700 applications. We aimed to discover a new voice, and luckily we have: an as-yet unpublished individual with experience of the tech industry and of academia.

"There were several proposals that the Board felt would make excellent books, but we think we have the best – and we hope that a really lively public debate will follow its publication. The issue it addresses is hugely important, and this is a new and thrilling way of starting such a discussion.”

The Nine Dots Prize Board is composed of ten internationally recognised and distinguished academics, authors, journalists and thinkers. They are:

  • Professor Diane Coyle – Professor of Economics at Manchester University, former Vice Chair of the BBC Trust and Economics Editor of the Independent
  • Professor Paul Gilroy – currently Professor of English at Kings College London, previously Giddens Professor of Social Theory at the London School of Economics
  • Professor Simon Goldhill (Chair) – Director of the Centre for Research in the Arts, Social Sciences and Humanities (CRASSH), Professor in Greek Literature and Culture and Fellow of King's College, Cambridge
  • E.J. Graff – Managing Editor of the Washington Post’s Monkey Cage blog and Senior Fellow at the Schuster Institute for Investigative Journalism at Brandeis University
  • Professor Alcinda Honwana – visiting Professor of Anthropology and International Development at the Open University and formerly was a program officer at the United Nations Office of the Special Representative for Children and Armed Conflict
  • Peter Kadas – Director of the Kadas Prize Foundation
  • Professor Ira Katznelson – President of the Social Science Research Council and former President of the American Political Science Association
  • Professor Roger Martin – Institute Director of the Martin Prosperity Institute and the Michael Lee-Chin Family Institute for Corporate Citizenship at the Rotman School of Management and the Premier’s Chair in Productivity & Competitiveness
  • Professor Riccardo Rebonato – Professor of Finance at EDHEC Business School, formerly Global Head of Rates and FX Research at PIMCO
  • Professor David Runciman – Professor of Politics and Head of the Department of Politics and International Studies at the University of Cambridge

James Williams, a 35-year-old doctoral candidate researching design ethics at Oxford University, has been announced as the inaugural winner of the $100,000 Nine Dots Prize at an awards ceremony at the British Library yesterday evening.

We had no idea whether we were reading the proposal of a professor, a novelist, a postman, a student or a lawyer.
Simon Goldhill

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

'Extreme sleepover #20' – welcome to dataworld

By lw355 from University of Cambridge - Big data. Published on Mar 31, 2017.

I’m standing 100 feet underground in a fluorescent-white room. In the centre, stand four rows of server cabinets. I’m following Matej, a data centre technician, as he carries out some diagnostic tests on the facility’s IT equipment. To get here we had to go through several security checks, including a high-tech biometric fingerprint scanner and a good old-fashioned, low-tech massive door.

The IT equipment is distributed over multiple floors, each going deeper and deeper below ground into the seemingly infinite depths of the data centre. There is a constant hum of electrical voltage down here; it’s the kind of vibratory, carcinogenic sound you would normally associate with pylon power lines and it makes you think you’re probably being exposed to some sort of brain-frying electric field. I ask Matej about this and he tells me, “it’s probably ok.”

When Matej is finished in this room we head downstairs. Our footsteps sound hollow and empty on the elevated metallic walkways. A complex highway of thick, encaged cables runs above our heads, along with large pipes circulating water around the data centre for cooling purposes. As we descend the galvanised steel stairway, it’s like boarding a spaceship that’s buried deep beneath the Earth’s surface.

The room we enter is almost completely white. The only other colour down here comes from thousands of server lights blinking rapidly like fireflies behind the electro-zinc-coated ‘Zintec’ doors of the server cabinets. We have entered the realm of data, an alien world of tiny, undulating lights that seem almost alive. These iridescent lights flash as data travels to and from the facility through fibre-optic cables at speeds of around 670 million miles per hour, close to the speed of light.

Take a walk inside a data centre with Alexander

This building has been designed with the sole purpose of providing optimal living conditions for data growth and survival. An ambient room temperature of around 20–21°C and a humidity level of 45–55% must constantly be maintained. In this sterile, dustless world of brushed metal surfaces, data live and thrive like precious crystals. Server cabinets become stalagmite formations sparkling frenetically with the digital activity of millions of people doing their daily things in that exact moment all around the world.

Virtually all our daily activity – both online and offline – entails the production of data, with 2.5 billion gigabytes of data being produced every 24 hours. This is stored in the 8.6 million data centres that have spread acoss the globe. Yet, few of us realise that we are using data centres.

Data centres now underpin an incredible range of activities and utilities across government, business and society, and we rely on them for even the most mundane activities: our electricity and water accounts are located in data centres, a single Google search can involve up to five data centres, information from the train tickets we swipe at turnstiles are routed through data centres. These places process billions of transactions every day and extreme efforts are made to ensure that they do not fail.

One such effort is the increasingly common practice of storing data underground in ‘disaster-proof’ facilities – in the same way that seed and gene banks store biological material that is essential for human survival. What does this say about the importance of data to our society? This is what I am down here researching. Working with data centres, IT security specialists, cloud computing companies and organisations that are trying to raise awareness about the vulnerabilities of digital infrastructures, I am exploring the cultural hopes, fears and imaginations of data as it pertains to what many are calling our ‘digital future’.

My fieldwork has led me to focus on the fears of disaster and technological failure that motivate data centre practices and discourses, from routine Disaster Recovery plans to storing hard drives in Faraday cages to protect them against electromagnetic threats. The current mass exodus into ‘the cloud’ is raising important questions about our increasing societal dependence upon digital technology and the resilience, sustainability and security of the digital infrastructure that supports our online and offline lives. Fears of a ‘digital’ disaster occurring in the future are also reflected in cultural artefacts such as TV shows about global blackouts and books about electromagnetic pulse events. In an age of constant and near compulsory connection to computers, tablets and smartphones, how would we survive if they all suddenly and simultaneously ceased to function?

 Data centres are being configured as infrastructures critical not only for supporting our data-based society, but also for backing up and even potentially re-booting ‘digital civilisation’, if it should collapse. My fieldwork is not all doom and disaster, though. In fact, sometimes it’s quite spectacular. Right now I am standing in a heavily air-conditioned aisle flanked on each side by large, monolithic cabinets of server racks.

“This is one of my favourite things,” Matej says, as he flicks the overhead lights off and plunges us into an abyssal darkness punctured only by server lights, flashing like phytoplankton all around us. For a moment, we watch these arrhythmic lights flickering, beautiful and important, some vanishingly small.

But these little lights have immense significance. Something huge is happening down here. It feels like you are witnessing something incomprehensibly vast, something so massively distributed, complex and connected to all of us that it’s hard to even know what you are seeing take place. It’s like looking at the stars.

Alexander is a PhD student at Fitzwilliam College with the Division of Social Anthropology. His research is supervised by Dr Christos Lynteris, and is funded by the Cambridge Home and EU Scholarship Scheme.

 

​Alexander Taylor provides a sensory snapshot of his fieldwork in high-security subterranean data centres exploring fears of technological failure in our data-dependent society.

There is a constant hum of electrical voltage down here; it’s the kind of vibratory, carcinogenic sound you would normally associate with pylon power lines and it makes you think you’re probably being exposed to some sort of brain-frying electric field
Alexander Taylor
Data centre

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

'Extreme sleepover #20' – welcome to dataworld

By lw355 from University of Cambridge - digital society. Published on Mar 31, 2017.

I’m standing 100 feet underground in a fluorescent-white room. In the centre, stand four rows of server cabinets. I’m following Matej, a data centre technician, as he carries out some diagnostic tests on the facility’s IT equipment. To get here we had to go through several security checks, including a high-tech biometric fingerprint scanner and a good old-fashioned, low-tech massive door.

The IT equipment is distributed over multiple floors, each going deeper and deeper below ground into the seemingly infinite depths of the data centre. There is a constant hum of electrical voltage down here; it’s the kind of vibratory, carcinogenic sound you would normally associate with pylon power lines and it makes you think you’re probably being exposed to some sort of brain-frying electric field. I ask Matej about this and he tells me, “it’s probably ok.”

When Matej is finished in this room we head downstairs. Our footsteps sound hollow and empty on the elevated metallic walkways. A complex highway of thick, encaged cables runs above our heads, along with large pipes circulating water around the data centre for cooling purposes. As we descend the galvanised steel stairway, it’s like boarding a spaceship that’s buried deep beneath the Earth’s surface.

The room we enter is almost completely white. The only other colour down here comes from thousands of server lights blinking rapidly like fireflies behind the electro-zinc-coated ‘Zintec’ doors of the server cabinets. We have entered the realm of data, an alien world of tiny, undulating lights that seem almost alive. These iridescent lights flash as data travels to and from the facility through fibre-optic cables at speeds of around 670 million miles per hour, close to the speed of light.

Take a walk inside a data centre with Alexander

This building has been designed with the sole purpose of providing optimal living conditions for data growth and survival. An ambient room temperature of around 20–21°C and a humidity level of 45–55% must constantly be maintained. In this sterile, dustless world of brushed metal surfaces, data live and thrive like precious crystals. Server cabinets become stalagmite formations sparkling frenetically with the digital activity of millions of people doing their daily things in that exact moment all around the world.

Virtually all our daily activity – both online and offline – entails the production of data, with 2.5 billion gigabytes of data being produced every 24 hours. This is stored in the 8.6 million data centres that have spread acoss the globe. Yet, few of us realise that we are using data centres.

Data centres now underpin an incredible range of activities and utilities across government, business and society, and we rely on them for even the most mundane activities: our electricity and water accounts are located in data centres, a single Google search can involve up to five data centres, information from the train tickets we swipe at turnstiles are routed through data centres. These places process billions of transactions every day and extreme efforts are made to ensure that they do not fail.

One such effort is the increasingly common practice of storing data underground in ‘disaster-proof’ facilities – in the same way that seed and gene banks store biological material that is essential for human survival. What does this say about the importance of data to our society? This is what I am down here researching. Working with data centres, IT security specialists, cloud computing companies and organisations that are trying to raise awareness about the vulnerabilities of digital infrastructures, I am exploring the cultural hopes, fears and imaginations of data as it pertains to what many are calling our ‘digital future’.

My fieldwork has led me to focus on the fears of disaster and technological failure that motivate data centre practices and discourses, from routine Disaster Recovery plans to storing hard drives in Faraday cages to protect them against electromagnetic threats. The current mass exodus into ‘the cloud’ is raising important questions about our increasing societal dependence upon digital technology and the resilience, sustainability and security of the digital infrastructure that supports our online and offline lives. Fears of a ‘digital’ disaster occurring in the future are also reflected in cultural artefacts such as TV shows about global blackouts and books about electromagnetic pulse events. In an age of constant and near compulsory connection to computers, tablets and smartphones, how would we survive if they all suddenly and simultaneously ceased to function?

 Data centres are being configured as infrastructures critical not only for supporting our data-based society, but also for backing up and even potentially re-booting ‘digital civilisation’, if it should collapse. My fieldwork is not all doom and disaster, though. In fact, sometimes it’s quite spectacular. Right now I am standing in a heavily air-conditioned aisle flanked on each side by large, monolithic cabinets of server racks.

“This is one of my favourite things,” Matej says, as he flicks the overhead lights off and plunges us into an abyssal darkness punctured only by server lights, flashing like phytoplankton all around us. For a moment, we watch these arrhythmic lights flickering, beautiful and important, some vanishingly small.

But these little lights have immense significance. Something huge is happening down here. It feels like you are witnessing something incomprehensibly vast, something so massively distributed, complex and connected to all of us that it’s hard to even know what you are seeing take place. It’s like looking at the stars.

Alexander is a PhD student at Fitzwilliam College with the Division of Social Anthropology. His research is supervised by Dr Christos Lynteris, and is funded by the Cambridge Home and EU Scholarship Scheme.

 

​Alexander Taylor provides a sensory snapshot of his fieldwork in high-security subterranean data centres exploring fears of technological failure in our data-dependent society.

There is a constant hum of electrical voltage down here; it’s the kind of vibratory, carcinogenic sound you would normally associate with pylon power lines and it makes you think you’re probably being exposed to some sort of brain-frying electric field
Alexander Taylor
Data centre

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Encouragement from teachers has greatest influence on less advantaged children

By fpjl2 from University of Cambridge - Big data. Published on Mar 28, 2017.

Schoolchildren who receive words of encouragement from a teacher are significantly more likely to continue their education beyond the age of 16 than those who do not, a new study suggests.

The influence of teacher encouragement appears to be much greater on students whose own parents never progressed past compulsory education – an important indicator of a less advantaged background.

For students from these backgrounds, encouragement increased entry into post-16 education from just over half to around two-thirds.  

The research also found that encouragement from a teacher has the greatest influence on those students most likely to be on the margin for university attendance.  

The University of Cambridge study used ‘big data’ techniques to look at the long-term impact of student-teacher rapport, and is the first to analyse the role it plays in university access. 

The findings, published in the journal Research in Higher Education, show that further education and social mobility policymaking might benefit from increased focus on the “relational aspects” of interactions between teachers and students. 

“Teachers are often relegated to course deliverers and classroom managers in the policy discussions around further education. However, it’s clear that teachers have more forms of influencing inequality than is currently appreciated,” said study author Dr Ben Alcott from Cambridge’s Faculty of Education.   

“When people speak of a positive school experience, they frequently cite a personal relationship with a teacher, and the encouragement they were given. Our research helps quantify that impact and show its significance, particularly for addressing social mobility.

“The importance of that teacher-student connection can get lost in the midst of exam statistics or heat of political debate.”

Some 4,300 adolescents in England were tracked from the age of thirteen onwards, completing a detailed questionnaire every year for the next seven years. During their last year of compulsory education, the students were asked whether a teacher had encouraged them to stay on in full-time education. 

Dr Alcott used mathematical modelling to “match” and compare students with similar attainment, experience and life histories – helping control for the effects these differences had. This makes it possible for the influence of teacher encouragement alone to be measured.

“This approach brings us plausibly close to reading the long-term effect of encouragement from teachers with the data we currently have available,” Alcott said. 

He found that, on average across all backgrounds and abilities, rates of entry into post-16 education were eight percentage points higher in students that reported receiving encouragement (74%) over those that said they did not (66%).

Based on previous examination scores (the UK’s SATs), teacher encouragement made the most difference for students with average academic achievement – those often on the verge of going either way when it comes to further education.

For Year 11 (or 10th grade) students in the middle third of results rankings, encouragement was linked to a 10 percentage-point increase in the likelihood of university entry, yet had no observable impact on students in the upper and lower thirds.    

The effect of teacher encouragement on students varied considerably depending on background – with the greatest difference seen for students with lower levels of parental education.

For students with parents who lacked any formal qualification, post-16 education enrolment increased 12 percentage points amongst those who received teacher encouragement (64%) compared with those who didn’t (52%).

This effect appeared to last into higher education, with that initial encouragement increasing the likelihood of university entry by 10 percentage points – one-fifth higher than students from similar backgrounds who did not report being encouraged.    

Students whose parents had some qualifications, but none past compulsory education, saw encouragement from teachers boost post-16 education by 13 percentage points (67% compared to 54%) and university entry by seven percentage points.

For those with parents who held university degrees, however, teacher encouragement mattered much less: increasing continued education by just six percentage points and making no difference at all to university attendance.

However, Alcott found that students from more advantaged backgrounds were likelier to report being encouraged by a teacher to stay in education.

For example, 22% of students receiving encouragement had a parent with a university degree, compared to 15% of those who did not. Similarly, students who do not report encouragement are a third more likely to have an unemployed parent (12% versus 9%).

Alcott, who formerly taught in a London academy school himself, says: “These results suggest that teachers themselves and the relationships they develop with students are real engines for social mobility.

“Many teachers take the initiative to encourage students in the hope they will progress in education long after they have left the classroom. It’s important that teachers know the effect their efforts have, and the children likely to benefit most.” 

‘Big data’ study finds that children from families with limited education have strongest long-term response to teacher encouragement, and are more likely to progress to university as a result.

The relationships teachers develop with students are real engines for social mobility.
Ben Alcott
West Somerset Community College at the Institute of Physics

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Enabling Civilian Witnesses to Report Human Rights Abuses

By ljlk2 from University of Cambridge - digital society. Published on Mar 08, 2017.

Social Media now provides a global platform for sharing first-hand ‘evidence’ of human rights violations. For civilian witnesses living in remote or isolated locations, social media may be the only lifeline open to them. But how can we distinguish authentic reports from propaganda, hoaxes and digital manipulation?

The Whistle, an automated tool being developed by a team at the University of Cambridge led by Dr Ella McPherson, aims to solve this problem by expediting the digital verification of reports.

“News is disseminated fast but verification is slow and often contested,” says McPherson. “For human rights NGOs, credibility can be lost in a moment if the evidence they are using for advocacy or in courts is later found to be false. No matter how devastating the documented violations, they cannot act on them unless they can verify them first.”

When so much is at stake, it is vital that NGOs and other international organisations are able to respond to authentic reports of human rights abuses, whilst ignoring the plethora of fraudulent evidence which is continually being produced. In order to do this NGO’s may employ ‘fact-finders’ to research and corroborate reports that come in via social media or online, by checking and cross-referencing metadata relating to location, time, weather, landscapes and digital footprints.

However, the sheer number of videos and photos purporting to document events is becoming overwhelming. When everything requires verification through the complex and lengthy process of metadata corroboration – the result is what McPherson has described as a ‘bottleneck’ that is disabling a significant source of potential witness accounts. 

The main issues at stake are that many civilian witnesses lack digital literacy, are unfamiliar with metadata and do not know that they should include it in reports. This makes evidence much harder to verify. Similarly, human rights fact-finders may also lack the knowledge necessary to accurately use online verification tools, which are complex and rapidly developing. Existing digital verification tools also require extensive manual cross-checking, which require a substantial amount of time and expertise. In addition to these problems, witnesses with existing digital footprints – organisational affiliations or a social media profile – are easier to verify. This can mean they are more easily heard than individuals with minimal resources or a negligible footprint online.

The Whistle is a digital platform that empowers both witnesses and fact-finders, by facilitating accurate reporting and speeding up the process of verification.

A mobile app simplifies the process of reporting for the witness, whilst simultaneously prompting them to include the metadata information required for verification. Aside from providing more metadata for the fact-finder to corroborate, The Whistle also serves to educate witnesses about what data is most helpful and why. It can also signpost them to sources of information and support around security and human rights.

On the receiving side, a ‘dashboard’ aggregates the reported information and automates a process of cross-checking, comparing social media reports, weather information and map databases, bringing together a variety of existing digital verification tools and practices in one place. It does this by using algorithmic indicators in addition to human input, a combination which is unusual in this context. The application automatically extracts metadata about the source, the location and the channel of a report (e.g. a video) as well cross-referencing specific frames against image databases to see if the report has previously appeared online in another guise.

The Whistle project was initially funded by the ESRC IAA, which enabled substantive market research that revealed a gap in the market. The IAA funding also facilitated the launch of a landing page website at www.thewhistle.org, and the development of a partial prototype. As work progressed, it became clear that the design and programming of the application would require more time and funding than originally anticipated. McPherson and her team subsequently partnered with Wikirate, an organization dedicated to eradicating corporate abuses through transparency about companies’ social and environmental impacts. Together with a number of other partners, Wikirate and The Whistle successfully applied for a three-year Horizon 2020 grant. The team has grown to six members and is currently working with NGO partners to develop and test a full prototype of The Whistle and to gain feedback from civilian witnesses. Looking forwards, the application plans to facilitate the reporting and verification of civilian witness accounts of human rights abuses in partnership with NGOs across the world.

We know from fact-finders that civilian witnesses do not necessarily know what metadata is or that they should include it with their information – even something as simple as panning the horizon for landmarks or turning on geolocation features.

Dr Ella McPherson

The Whistle provides human rights researchers, NGO’s and international organisations with a wealth of cross-referenced information, reducing the time and digital expertise currently required to verify digital witness accounts of human rights violations.

Viral charity campaigns have a psychological 'recipe' and all-too-brief lifespan

By fpjl2 from University of Cambridge - digital society. Published on Feb 13, 2017.

A University of Cambridge researcher has identified a recipe for the new breed of wildly successful online charity campaigns such as the ALS Ice Bucket Challenge – a phenomenon he has labelled “viral altruism” – and what might make them stick in people’s minds.    

However, he says the optimistic use of global digital networks to propel positive social change is balanced by the shallow, short-lived nature of engagement with anything viral.

Writing in the journal Nature Human Behaviour, social psychologist Dr Sander van der Linden has outlined the key psychological levers he says underpin the new wave of viral altruism that is increasingly taking over our Facebook feeds.

These include the power of social norms, particularly the appeal of joining a social consensus and the desire to conform to prosocial behaviour (such as appearing charitable), having a clear moral incentive to act, and the appetite for a ‘warm glow’: the positive emotional benefit derived from feeling compassionate.

One of the most important ingredients – and the hardest to achieve – is ‘translational impact’: the conversion of online token support, or ‘clicktivism’, into sustained real world contributions, whether financial donations or a long-term commitment to an issue.   

This, he says, involves a shift in motivation from the ‘extrinsic’ – incentives conditional on outside social pressures – to the ‘intrinsic’: an incentive that has been internalised to become a “new personal normal” for an individual.

Part of van der Linden’s initial research has been to pull together data such as Google and Wikipedia searches as well as donations to indicate the longevity and engagement levels of the ALS Ice Bucket Challenge campaign. 

The Challenge reached unprecedented ‘virality’ during August 2014. The formula of videoing ice-cold water being poured over your head and posting it to social media while publicly nominating others to do the same in support of a motor neurone disease charity reached approximately 440 million people worldwide, with over 28 million joining in.  

'Brightly but briefly'

Yet van der Linden found that the Challenge burned brightly but briefly: with online interest and donations reverting to pre-viral levels in mere weeks. The engagement was also superficial: estimates suggest that 1 in 4 participants did not mention the ALS charity in their videos and only 1 in 5 mentioned a donation.

And, while the 2014 campaign caused a significant spike in donations – some $115m – when the ALS charity attempted to reboot the Ice Bucket Challenge the following year it raised less than 1% of the previous summer.

Other examples of viral altruism considered to be successful also appear to have an equally brief “half-life”. The Facebook organ donor initiative elicited more than 60% of its total online registrations in the first two days before numbers rapidly dropped off. Save Darfur was one of the largest campaigns on Facebook; after joining, most members never donated money or recruited anyone else.

Van der Linden believes converting the brief social pressures of viral altruism into self-sustaining personal motivations is the key to leveraging new digital networks for long-term engagement with the big issues of our time, such as climate change.

However, he argues that it may be the very viral nature of ‘viral altruism’ that acts as a barrier to this.

“Society now has the ability to connect and mobilise over a billion Facebook users to action on specific social issues in a fast and low-cost manner, but it is becoming clear this entails viral phenomena which by their very nature are ephemeral and superficial,” says van der Linden, from Cambridge’s Department of Psychology. 

Hyper-viral paradox

“Just as a flame that burns twice as bright burns half as long, so a rapid social consensus spike reaches an equally rapid saturation point.

“Once the social tipping point of a campaign has passed, momentum can decay quickly and the purpose can get diluted. Once the ALS campaign had reached peak virality, many people were just pouring cold water over their heads without necessarily referencing the charity.

“Paradoxically, increasing meaningful engagement through viral altruism might actually require deliberately hindering the hyper-viral nature at some point with a stabilising force. Perhaps introducing aspects to a campaign that increasingly require more commitment – slowing growth and encouraging deeper engagement. If we want people to internalise a new normal, we need to give them a window big enough to do that.

“Deeper engagement seems especially vital. Something as simple as a single phrase connecting a campaign to its cause can make a difference. For example, those who mentioned the ALS charity in their Ice Bucket Challenge video were five times more likely to donate money than those who did not.”

SMART recipe

Van der Linden has set out his recipe for viral altruism using the acronym SMART: Social influences; Moral imperatives; Affective Reactions; Translational impact.

The ALS campaign managed to exploit a two-pronged approach to 'social influences'. People were influenced by the example of those in their network, and wanted to join the burgeoning consensus. The nature of the campaign also meant that many were publicly challenged to participate by their social network, and risked the 'social sanction' of being seen to lack compassion if they then didn't.

Helping people with a debilitating disease was seen as a 'moral imperative'. Van der Linden says that having 'identifiable victims' such as scientist Prof Stephen Hawking allowed people to relate to the cause.

Campaigns that allow for the creation of a shared identity between the individual and the cause over time appear to be more successful in achieving translational impact.

Sander van der Linden

'Affective Reactions' is the response to strong emotional content. "Empathy is an emotional contagion," says van der Linden. "We are evolutionarily hard-wired to 'catch' other people's feelings. Responding with an altruistic act give us a 'warm glow' of positivity. Similarly, people often respond to social injustice, such as genocide, with strong moral outrage."

However, where almost all campaigns stumble is 'Translational impact', he says. "Extrinsic incentives, such as competitions or network pressure, can actually undermine people's intrinsic motivation to do good by eroding moral sentiment. Motivation to participate can get sourced from a desire to 'win' a challenge or appear virtuous rather than caring about the cause itself."

Climate change is an example of a major global issue that currently scores pretty much zero for the SMART recipe, says van der Linden.

"Climate change often fails to elicit strong emotional engagement, there is little to no societal pressure to act on climate change in our daily lives, most people do not view it as a fundamental moral issue, and the long-term nature of the problem requires more than a one-off donation."

He suggests that using the SMART recipe could be a way to reverse engineer more effective climate change campaigns that harness viral altruism, but the problem of translating impact remains.

One of the more impactful campaigns van der Linden highlights is 'Movember': the month-long growing of a moustache to raise awareness of men's health. Starting with just 30 people in 2003, the campaign didn't experience viral hypergrowth, but developed over years to reach about 5 million members by 2014 - by which time the charity reported 75% of participants were more aware of health issues facing men.

"Campaigns that allow for the creation of a shared identity between the individual and the cause over time appear to be more successful in achieving translational impact."

New work focusing on the ALS Ice Bucket Challenge reveals very brief shelf life of such viral campaigns, and suggests the nature of ‘virality’ and social tipping points themselves may be a stumbling block to deeper engagement with social issues that campaigns aim to promote.    

Increasing meaningful engagement through viral altruism might actually require deliberately hindering the hyper-viral nature at some point with a stabilising force
Sander van der Linden
ALS Ice Bucket Challenge

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

How bright is your digital future?

By Anonymous from University of Cambridge - Big data. Published on Jan 18, 2017.

The combination of new technologies, IT infrastructures and data analytics holds out an alluring possibility of a world in which the end-to-end supply chain is utterly transformed – highly connected, flexible, efficient, resilient and truly responsive to customer needs. Each of those attributes sounds attractively incremental but put them together and you have a completely new way of doing business and one in which customers are not just on the receiving end of a product or service but are central to it.

A good example of this is the pharmaceutical sector. As part of the REMEDIES project, we are working with the major players in the UK pharmaceutical supply chain to address some of the challenges they face, such as tackling the hundreds of days’ of inventory sitting in the supply chain and the vast quantities of waste caused by patients not taking the drugs they are prescribed.

Using digital technologies and data-rich systems to make the pharmaceutical supply chain much more efficient is one thing but we are also mapping an entirely new business model in which drugs can be manufactured to order – possibly at the local pharmacy. Not only would this meet a patient’s individual medical needs, but the consumption and effects of those drugs can be continuously monitored to help doctors better support their patients.

A brave new world, in other words, of personalised medicine enabled by digital manufacturing processes, digital infrastructures and lots of data. But realising this vision of a digital future remains elusive, particularly for the largest global businesses.

Many of these companies recognise the need to digitalise aspects of their supply chain, often in response to particular challenges. They may, for example, as in the pharmaceutical sector, have a pressing need to solve the intransigent inventory management issues that bedevil many supply chains. They may have an issue with quality and see digitalisation as the best way to ensure their products are of a consistently high quality and their provenance is traceable.

Or they may be losing competitive advantage through poor customer service and see a digital agenda as a way of regaining market share, possibly while supporting their ambitions to reduce environmental impact.

But developing an end-to-end digital supply chain involves a major transformation both at a conceptual level and in execution. And while thought leaders and change agents within big companies may see the prize, CEOs and shareholders will be much more cautious given the levels of investment and organisation-wide disruption it entails. This is particularly the case for the global giants with a history of merger and acquisition (M&A) and an array of legacy systems to integrate. Even without the complication of M&A, all large companies have to organise themselves into manageable structures, which have a natural tendency to turn into silos and hence become obstacles to organisational change.

There is also the wider question of a lack of digital skills and attitudes across the board – at senior and middle management levels as well as within day-to-day factory operations. Companies may be able to see the opportunity, acquire the technology and capture the data but a shortage of both skills and mindset presents a significant barrier.

One of the challenges with the digital supply chain vision is the sheer scale and ambition of it. At the Centre for International Manufacturing, we have begun to conceptualise what a digital supply chain might look like and break it down into key areas to help companies understand the key ways in which digitalisation can impact on their organisation. We have been doing this by talking to companies both individually and as a non-competitive group.

Having identified the key areas, we have been developing ‘maturity models’ against which companies can benchmark their current performance, identify where the greatest opportunities lie and start to think about where to prioritise their efforts.

Factory design and production processes

Digital developments in factory design and production processes underpin the extended supply chain. The flexible factory is an important concept in this rapidly moving environment: how can you design and configure a factory for technologies which you don’t yet know? In this context, factories need to be modular and reconfigurable. One of the questions our framework helps companies consider is this: it is relatively straightforward to design a state-of-the-art, highly flexible plug-and-play factory – but is it cost-effective? Is it where companies will be able to create and capture most value?

Making the most of data

Some companies are already very good at gathering product and customer data but the challenge is how to integrate that data and use it to make better decisions about, for example, product lifecycle management, sales forecasting and designing products and services in response to customer needs. Data ownership is fast becoming an important issue in the supply chain and service delivery context. When partners are involved, who owns and can access the data is a critical question. Data sharing and connectivity also raises the question of open source versus ‘black box’ and developing common international data standards across sectors. In this area we must also consider the resilience of these digital supply chains and understand the cyber security challenges they may present.

Flexibility versus connectivity

One of the conceptual and practical challenges for organisations is whether to build monolithic, enterprise-wide systems that can connect supply chains. Clearly, for many companies – particularly those with a history of M&A – it would require a huge act of organisational will, not to mention significant investment, to move to a common platform. And, would doing so actually deliver a sufficiently flexible and reconfigurable solution? Instead, companies are talking about developing a ‘digital backbone’ that can interface with other systems to provide more networked and flexible approaches to optimising the end-to-end supply chain. And this digital backbone is more than an IT system – it should embody the critical touch points and interfaces between organisations as well as the data architectures and analytics. It also signifies a cultural shift to digital.

The last leg

Using web-based systems to fulfil orders and manage the complexity of last-mile logistics is something that we have seen business-to-consumer companies do with impressive levels of sophistication and achieve corresponding levels of competitive advantage. For many large manufacturers there is still work to be done in developing systems that can support product delivery to multiple points of sale and ultimately direct to the end customer. But the opportunities are clear and create a virtuous circle. By delivering better customer service you not only attract new customers (and retain the old ones) but you also get access to better customer data which in turn can improve both the product and the service you offer.  There are also many efficiencies to be had from digitalising this last leg of the supply chain though better stock management and reduced transport costs.

Towards the digital supply chain

By breaking down the digital supply chain into distinct but connected scenarios against which companies can measure their performance and aspirations, we believe we have created a powerful framework that will help them develop their digital supply chain capabilities. The scenarios help to clarify thinking and develop a strategic approach to digitalisation which is both deliverable and will create maximum value for the company.

The next step is to put the strategy into action. 

First published in IfM Review.  

Dr Jag Srai, Head of the Centre for International Manufacturing at Cambridge's Institute for Manufacturing, and colleagues are developing new ways to help companies embrace the challenges and opportunities of digitalising the extended supply chain. Here, he provides a glimpse of this digital future.

A brave new world of personalised medicine enabled by digital manufacturing processes, digital infrastructures and lots of data
Jag Srai
pharmaceuticals

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

How bright is your digital future?

By Anonymous from University of Cambridge - digital society. Published on Jan 18, 2017.

The combination of new technologies, IT infrastructures and data analytics holds out an alluring possibility of a world in which the end-to-end supply chain is utterly transformed – highly connected, flexible, efficient, resilient and truly responsive to customer needs. Each of those attributes sounds attractively incremental but put them together and you have a completely new way of doing business and one in which customers are not just on the receiving end of a product or service but are central to it.

A good example of this is the pharmaceutical sector. As part of the REMEDIES project, we are working with the major players in the UK pharmaceutical supply chain to address some of the challenges they face, such as tackling the hundreds of days’ of inventory sitting in the supply chain and the vast quantities of waste caused by patients not taking the drugs they are prescribed.

Using digital technologies and data-rich systems to make the pharmaceutical supply chain much more efficient is one thing but we are also mapping an entirely new business model in which drugs can be manufactured to order – possibly at the local pharmacy. Not only would this meet a patient’s individual medical needs, but the consumption and effects of those drugs can be continuously monitored to help doctors better support their patients.

A brave new world, in other words, of personalised medicine enabled by digital manufacturing processes, digital infrastructures and lots of data. But realising this vision of a digital future remains elusive, particularly for the largest global businesses.

Many of these companies recognise the need to digitalise aspects of their supply chain, often in response to particular challenges. They may, for example, as in the pharmaceutical sector, have a pressing need to solve the intransigent inventory management issues that bedevil many supply chains. They may have an issue with quality and see digitalisation as the best way to ensure their products are of a consistently high quality and their provenance is traceable.

Or they may be losing competitive advantage through poor customer service and see a digital agenda as a way of regaining market share, possibly while supporting their ambitions to reduce environmental impact.

But developing an end-to-end digital supply chain involves a major transformation both at a conceptual level and in execution. And while thought leaders and change agents within big companies may see the prize, CEOs and shareholders will be much more cautious given the levels of investment and organisation-wide disruption it entails. This is particularly the case for the global giants with a history of merger and acquisition (M&A) and an array of legacy systems to integrate. Even without the complication of M&A, all large companies have to organise themselves into manageable structures, which have a natural tendency to turn into silos and hence become obstacles to organisational change.

There is also the wider question of a lack of digital skills and attitudes across the board – at senior and middle management levels as well as within day-to-day factory operations. Companies may be able to see the opportunity, acquire the technology and capture the data but a shortage of both skills and mindset presents a significant barrier.

One of the challenges with the digital supply chain vision is the sheer scale and ambition of it. At the Centre for International Manufacturing, we have begun to conceptualise what a digital supply chain might look like and break it down into key areas to help companies understand the key ways in which digitalisation can impact on their organisation. We have been doing this by talking to companies both individually and as a non-competitive group.

Having identified the key areas, we have been developing ‘maturity models’ against which companies can benchmark their current performance, identify where the greatest opportunities lie and start to think about where to prioritise their efforts.

Factory design and production processes

Digital developments in factory design and production processes underpin the extended supply chain. The flexible factory is an important concept in this rapidly moving environment: how can you design and configure a factory for technologies which you don’t yet know? In this context, factories need to be modular and reconfigurable. One of the questions our framework helps companies consider is this: it is relatively straightforward to design a state-of-the-art, highly flexible plug-and-play factory – but is it cost-effective? Is it where companies will be able to create and capture most value?

Making the most of data

Some companies are already very good at gathering product and customer data but the challenge is how to integrate that data and use it to make better decisions about, for example, product lifecycle management, sales forecasting and designing products and services in response to customer needs. Data ownership is fast becoming an important issue in the supply chain and service delivery context. When partners are involved, who owns and can access the data is a critical question. Data sharing and connectivity also raises the question of open source versus ‘black box’ and developing common international data standards across sectors. In this area we must also consider the resilience of these digital supply chains and understand the cyber security challenges they may present.

Flexibility versus connectivity

One of the conceptual and practical challenges for organisations is whether to build monolithic, enterprise-wide systems that can connect supply chains. Clearly, for many companies – particularly those with a history of M&A – it would require a huge act of organisational will, not to mention significant investment, to move to a common platform. And, would doing so actually deliver a sufficiently flexible and reconfigurable solution? Instead, companies are talking about developing a ‘digital backbone’ that can interface with other systems to provide more networked and flexible approaches to optimising the end-to-end supply chain. And this digital backbone is more than an IT system – it should embody the critical touch points and interfaces between organisations as well as the data architectures and analytics. It also signifies a cultural shift to digital.

The last leg

Using web-based systems to fulfil orders and manage the complexity of last-mile logistics is something that we have seen business-to-consumer companies do with impressive levels of sophistication and achieve corresponding levels of competitive advantage. For many large manufacturers there is still work to be done in developing systems that can support product delivery to multiple points of sale and ultimately direct to the end customer. But the opportunities are clear and create a virtuous circle. By delivering better customer service you not only attract new customers (and retain the old ones) but you also get access to better customer data which in turn can improve both the product and the service you offer.  There are also many efficiencies to be had from digitalising this last leg of the supply chain though better stock management and reduced transport costs.

Towards the digital supply chain

By breaking down the digital supply chain into distinct but connected scenarios against which companies can measure their performance and aspirations, we believe we have created a powerful framework that will help them develop their digital supply chain capabilities. The scenarios help to clarify thinking and develop a strategic approach to digitalisation which is both deliverable and will create maximum value for the company.

The next step is to put the strategy into action. 

First published in IfM Review.  

Dr Jag Srai, Head of the Centre for International Manufacturing at Cambridge's Institute for Manufacturing, and colleagues are developing new ways to help companies embrace the challenges and opportunities of digitalising the extended supply chain. Here, he provides a glimpse of this digital future.

A brave new world of personalised medicine enabled by digital manufacturing processes, digital infrastructures and lots of data
Jag Srai
pharmaceuticals

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Frankly, do we give a damn…? Study finds links between swearing and honesty

By ps748 from University of Cambridge - Big data. Published on Jan 16, 2017.

Profanity is obscene language which, in some social settings is considered inappropriate and unacceptable. It often refers to language that contains sexual references, blasphemy or other vulgar terms. It’s usually related to the expression of emotions such as anger, frustration or surprise. But profanity can also be used to entertain and win over audiences.

There are conflicting attitudes to profanity and its social impact has changed over the decades. In 1939, Clark Gable uttering the memorable line “Frankly my dear, I don’t give a damn” in the film Gone with the Wind, was enough to land the producers a $5,000 fine. Nowadays our movies, TV shows and books are peppered with profane words and, for the most part, we are more tolerant of them.

As dishonesty and profanity are both considered deviant they are often viewed as evidence of low moral standards. On the other hand, profanity can be positively associated with honesty. It is often used to express unfiltered feelings and sincerity. The researchers cite the example of President-elect Donald Trump who used swear words in some of his speeches while campaigning in last year’s US election and was considered, by some, to be more genuine than his rivals.

Dr David Stillwell, a lecturer in Big Data Analytics at the University of Cambridge, and a co-author on the paper, says: “The relationship between profanity and dishonesty is a tricky one. Swearing is often inappropriate but it can also be evidence that someone is telling you their honest opinion. Just as they aren’t filtering their language to be more palatable, they’re also not filtering their views. ”

The international team of researchers set out to gauge people’s views about this sort of language in a series of questionnaires which included interactions with social media users.

In the first questionnaire 276 participants were asked to list their most commonly used and favourite swear words. They were also asked to rate their reasons for using these words and then took part in a lie test to determine whether they were being truthful or simply responding in the way they thought was socially acceptable. Those who wrote down a higher number of curse words were less likely to be lying.

A second survey involved collecting data from 75,000 Facebook users to measure their use of swear words in their online social interactions. The research found that those who used more profanity were also more likely to use language patterns that have been shown in previous research to be related to honesty, such as using pronouns like “I” and “me”. The Facebook users were recruited from across the United States and their responses highlight the differing views to profanity that exist between different geographical areas. For example, those in the north-eastern states (such as Connecticut, Delaware, New Jersey and New York) were more likely to swear whereas people were less likely to in the southern states (South Carolina, Arkansas, Tennessee and Mississippi).

Reference

Gilad Feldman et al “Frankly, we do give a damn: The relationship between profanity and honesty” DOI:10.1177/1948550616681055

 

It’s long been associated with anger and coarseness but profanity can have another, more positive connotation. Psychologists have learned that people who frequently curse are being more honest. Writing in the journal Social Psychological and Personality Science a team of researchers from the Netherlands, the UK, the USA and Hong Kong report that people who use profanity are less likely to be associated with lying and deception.

Swearing is often inappropriate but it can also be evidence that someone is telling you their honest opinion. Just as they aren't filtering their language to be more palatable, they're also not filtering their views
David Stillwell
Swear word

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Pain in the machine: a Cambridge Shorts film

By amb206 from University of Cambridge - digital society. Published on Nov 02, 2016.

Pain is vital: it is the mechanism that protects us from harming ourselves. If you put your finger into a flame, a signal travels up your nervous system to your brain which tells you to snatch your finger away. This response isn’t as simple as it sounds: the nervous system is complex and involves many areas of the brain.

We’re developing increasingly sophisticated machines to work for us. In the future, robots might live alongside us as companions or carers. If pain is an important part of being human, and often keeps us safe, could we create a robot that feels pain?  These ideas are explored by Cambridge researchers Dr Ewan St John Smith and Dr Beth Singler in their 12-minute film Pain in the Machine.

Already we have technologies that respond to distances and touch. A car, for example, can detect and avoid an object; lift doors won’t shut on your fingers. But although this could be seen as a step towards a mechanical nervous system, it isn’t the same as pain. Pain involves emotion. Could we make machines which feel and show emotion – and would we want to?

Unpleasant though it is, pain has sometimes been described as the pinnacle of human consciousness. The human capacity for empathy is so great that when a robotics company showed film clips of robots being pushed over and kicked, views responded as if the robots were being bullied and abused. Pain is both felt and perceived.

Movies have imagined robots with their own personalities – sometimes cute but often evil. Perhaps the future will bring robots capable of a full range of emotions. These machines might share not only our capacity for pain but also for joy and excitement.

But what about the ethical implications? A new generation of emotionally-literate robots will, surely, have rights of their own

Pain in the Machine is one of four films made by Cambridge researchers for the 2016 Cambridge Shorts series, funded by Wellcome Trust ISSF. The scheme supports early career researchers to make professional quality short films with local artists and filmmakers. Researchers Beth Singler (Faculty of Divinity) and Ewan St John Smith (Department of Pharmacology) collaborated with Colin Ramsay and James Uren of Little Dragon Films.

The pain we experience as humans has physical and emotional components. Could we develop a machine that feels pain a similar way – and would we want to? The first of four Cambridge Shorts looks at the possibilities and challenges.

Still from Pain in the Machine

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Protecting our data and identity: how should the law respond?

By ts657 from University of Cambridge - digital society. Published on Oct 28, 2016.

The freedom of expression and the need for privacy may be strange bedfellows today – but could full-blown estrangement beckon in a digital future that makes the leap from user-controlled content to unfiltered, online sharing of, well, everything?

A future where streaming your life online becomes the norm is not unthinkable, according to Dr David Erdos, whose research in the Faculty of Law explores the nature of data protection. “Take something like Snapchat Spectacles or Google Glass,” he says. “Such technology could very quickly take off, and all of a sudden it becomes ‘normal’ that everyone is recording everything, both audibly and visually, and the data is going everywhere and being used for all sorts of purposes – some individual, some organisational.”

This makes questions about what control we have over our digital footprint rather urgent.

“You can see that we need to get some grip on how the right to privacy can be enforced as technologies continue to develop that can pose serious threats to individuals’ sense of dignity, reputation, privacy and safety,” he adds.

One enforcement Erdos refers to is Google Spain, a ruling made in 2014 by the Court of Justice of the European Union (CJEU) that examined search engines’ responsibilities when sharing content about us on the world wide web.

The CJEU ruled that people across all of the 28 EU Member States have a ‘right to be forgotten’ online, giving them an ability to prohibit search engines indexing inadequate, irrelevant or other illegal information about them against their name. This right to be forgotten is based on Europe’s data protection laws and applies to all online information about a living person.

Google responded by publishing a form you can submit to have such links to content (not the actual content) removed. I put it to the test – Google refuses on the basis that web links to my long-closed business are ‟justified” as they ‟may be of interest to potential or current consumers”.

Erdos explains that data protection doesn’t always work as it was originally intended to. “On paper, the law is in favour of privacy and the protection of individuals – there are stringent rules around data export, data transparency and sensitive data, for example.

“But that law was in essence developed in the 1970s, when there were few computers. Now we have billions of computers, and the ease of connectivity of smartphones and the internet.  Also, sharing online is not practically constrained by EU boundaries.

“That means the framework is profoundly challenged. There needs to be a more contextual legal approach, where the duties and possibly also the scope take into account risk as well as the other rights and interests that are engaged.  That law must then be effectively enforced.”

In fact, the EU data protection law currently extends surprisingly far. “By default, the law regulates anyone who alone, or jointly with others, does anything with computerised information that mentions a living person,” Erdos explains. “That could include many individuals on social networking sites. If you’re disseminating information about a third party to an indeterminate number of people, you’re (in theory at least) responsible for adherence to this law.”

Tweeters, for instance, may have to respond to requests for data (Tweets) to be rectified for inaccuracy or even removed entirely, and field ‘subject access requests’ for full lists of everything they’ve Tweeted about someone. And under the new General Data Protection Regulation that comes into effect in 2018, the maximum penalty for an infringement is €20 million (or, in the case of companies, up to 4% of annual global turnover).

When it comes to search engines or social media, Erdos admits that a strict application of the law is “not very realistic”. He adds: “There’s a systemic problem in the gap between the law on the books and the law in reality, and the restrictions are not desperately enforced.”

Erdos believes inconsistencies in the law could be exploited online by the ruthless. “The very danger of all-encompassing, stringent laws is that it seems as if responsible organisations and individuals who take them seriously are hamstrung while the irresponsible do whatever they want.”

This also applies to ‘derogations’ – areas where the law instructs a balance must be struck between data protection and the rights to freedom of journalistic, literary and artistic expression.

“Member states have done radically different things in their formal law here – from nothing at all through to providing a blanket exception – neither of which was the intention of the EU scheme.”

As the new law in 2018 will empower regulators to hand out fines of up to hundreds of millions of euros to large multinational companies, Erdos is passionate about the urgency of Europe getting a coordinated and clear approach on how its citizens can exercise their data protection rights.

“We are giving these regulators quite enormous powers to enforce these rules and yet do we have a good understanding of what we want the outcome to be and what we’re expecting individuals and organisations to do?” Erdos ponders.

“To me, this means that the enforcement will become more and more important. Data protection is not just a technical phrase – people really do need protection. The substance of the law needs to be hauled into something that’s more reasonable. That protection needs to be made real.”

Erdos’ research also explores the nature of data protection and academic freedom, and he successfully argued for academic expression to be added to the list of free speech derogations in the 2018 legislation. “I have come across the most egregious examples of research guidance stipulating alleged data protection requirements, including claims that published research can’t include any identifiable personal data at all,” says Erdos.

“In a survey of EU data protection authorities, I asked whether a journalist’s undercover investigation into extremist political beliefs and tactics and an academic’s undercover research into widespread claims of police racism could be legal under data protection. Not one regulator said the activity of the journalist would in principle be illegal, but almost half said the academic’s activity would be unlawful.

 “Academics aim to write something of public importance, and make it rigorous. The old law was seen to prioritise even tittle-tattle in a newspaper over academic research; one hopes this will largely be removed by the new law.”

For many, the greatest concern remains the potential threats to their privacy. In order for consumers to feel safe with emerging technology, law makers may have to legislate for potential breaches now, rather than react after the damage is done.

“We don’t want to respond in a panic of registering or documenting everything, but the alternative of collapse into an ‘anything goes’ situation is equally dangerous.

“Apps like Snapchat show many people value being able to upload certain pictures and information that soon disappear. We don’t want people forgetting what they’re sharing today, and then worrying far too late how third parties are using that information.”

Would Erdos himself ever use Snapchat Spectacles or Google Glass (he does own a smartphone)? He laughs. “Let’s face it, email, the internet, Google search… people ended up having to use them. So, never say never!”

Many of us see our privacy as a basic right. But in the digital world of app-addiction, geolocation tracking and social oversharing, some may have cause to wonder if that right is steadily and sometimes willingly being eroded away.

You can see that we need to get some grip on how the right to privacy can be enforced as technologies continue to develop that can pose serious threats to individuals’ sense of dignity, reputation, privacy and safety
David Erdos
Banksy stencil

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Elvis is alive and the Moon landings were faked: the (conspiracy) theory of everything

By sjr81 from University of Cambridge - digital society. Published on Oct 25, 2016.

Elvis is alive, the Moon landings were faked and members of the British Royal Family are shapeshifting lizards.

Not only that: 9/11 was an inside job, governments are deliberately concealing evidence of alien contact, and we are all being controlled by a sinister, shadowy cartel of political, financial and media elites who together form a New World Order.

As a global population we are awash with conspiracy theories. They have permeated every major event, across every level of society; from the French Revolution to the War on Terror. In doing so, they have attracted devotees in their millions; from lone survivalists to presidential nominees such as Donald Trump – who claimed Ted Cruz’s father had links to Lee Harvey Oswald and, by inference, to the murder of President John F. Kennedy.

But what effects do conspiracy theories really have on the public as we go about our day-to-day lives? Are they merely harmless flights of fancy propagated by those existing on the margins of society, or is their reach altogether more sinister? Do runaway conspiracy theories influence politicians, decision-makers and, by extension, the public at large? And what effect has the advent of the internet and mass, instant communication across social media platforms had on the spread of conspiracy theories around the world?

Since 2013, a team of Cambridge researchers and visiting fellows has been examining the theories and beliefs about conspiracies that have become such an enduring feature of modern society. Conspiracy and Democracy: History, Political Theory and Internet Research is a five-year, interdisciplinary research project based at CRASSH (Centre for Research in the Arts, Social Sciences and Humanities) and funded by the Leverhulme Trust.

The project brings together historians, political theorists, philosophers, anthropologists and internet engineers as it seeks to understand what additional factors must be at work for conspiracy theories to enjoy such prevalence in the 21st century.

Professor John Naughton who, along with Professor Sir Richard Evans and Professor David Runciman, is one of the three project directors, explains: “Studying conspiracy theories provides opportunities for understanding how people make sense of the world and how societies function, as well as calling into question our basic trust in democratic societies.

“Our project examines how conspiracies and conspiracy theorising have changed over the centuries and what, if any, is the relationship between them? Have conspiracy theories appeared at particular moments in history, and why?

“We wanted to counter the standard academic narrative that conspiracy theories are beneath contempt. We were anxious to undertake a natural history of theorising, to study it seriously from a 21st-century context.”

Despite the onset of the digital age, Naughton and his colleagues do not believe that the internet has necessarily increased the influence of conspiracy theories on society as a whole. Indeed, research suggests that although the spread of conspiracy theories is often instantaneous in the digital world, so too is the evidence to debunk them.

Likewise, the team’s work so far suggests that online, as in life, we largely surround ourselves with people of like-minded views and opinions, effectively partitioning ourselves from a diversity of world views.

“The internet doesn’t make conspiracy theories more persuasive, it actually seems to compartmentalise people,” adds Naughton. “We more efficiently come into contact with those who hold similar views, but we also mostly end up working in echo chambers. That’s the way the internet works at the moment – especially in social media: you end up somewhere where everyone has the same views.

“The effect is a more concentrated grouping of opinions, and that’s the same for everything else, not just conspiracy theories. I follow 800 people on Twitter. Not one of them celebrated Brexit. I was in an echo chamber.”

Dr Alfred Moore, a postdoctoral researcher on the project, adds: “The question of the effect of the internet is a really interesting one. How far can the emergence and success of today’s populist movements be explained in terms of technological changes and especially social media? My first instinct is to say a little bit, but probably not much.

“Technologies have made it less costly to communicate, which means it’s easier to find, talk to and organise supporters without the financial and organisational resources of political parties. Both Corbyn and Trump make heavy use of social media as an alternative to a supposedly biased ‘mainstream’ media and the influence of their parties. It also demonstrates how the internet can promote polarisation by making it easy for people to find information they agree with and to filter out everything else.”

For those reasons, Naughton and Moore believe that some of the most famous conspiracy theories – such as David Icke’s theories about shapeshifting reptiles or feverish claims about the death of Princess Diana – are not particularly dangerous as they don’t appear to generate tangible actions or outcomes in the real world. In fact, the Conspiracy and Democracy team question whether these silos effectively disable the capacity for many conspiracy theories to take a firm hold in the public consciousness or threaten our democratic processes.

“A lot remains to be done in researching the history, structure and dynamics of conspiracy theories, their relationships with real conspiracies, and the changes they have undergone through time,” adds Evans. “You might think that conspiracy theories cause anxiety and depression among ordinary people, and undermine trust in our political institutions and the people who run them, but there are plenty of other reasons for this lack of trust apart from conspiracy theories.

“The debate goes on, but it’s not a case of conspiracy theories threatening democracies. By themselves, such theories may reinforce political suspicion and prejudice but they’re not the origin of it. On the whole, I think it’s fair to conclude that the scale of the threat is pretty limited.

“Some varieties, like antisemitism, can cause huge damage, but others are pretty harmless. Does it really matter that some people think the moon landings were faked? In the end, few people believe we are ruled by alien lizards.”

As a global population we are awash with conspiracy theories. But what effect do these really have on the public as we go about our day-to-day lives, asks a team of Cambridge researchers.

The internet doesn’t make conspiracy theories more persuasive, it actually seems to compartmentalise people
John Naughton
Moon1

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Combating cybercrime when there's plenty of phish in the sea

By sc604 from University of Cambridge - digital society. Published on Oct 21, 2016.

We’ve all received the emails, hundreds, maybe thousands of them. Warnings that our bank account will be closed tomorrow, and we’ve only got to click a link and send credit card information to stop it from happening. Promises of untold riches, and it will only cost a tiny fee to access them. Stories of people in desperate circumstances, who only need some kind soul to go to the nearest Western Union and send a money transfer to save them.

Tricking people into handing over sensitive information such as credit card details – known as ‘phishing’ – is one of the ways criminals scam people online. Most of us think we’re smarter than these scams. Most of us think that we could probably con the con artist if we tried. But we would be wrong.

Across the world, cybercrime is booming. When the UK government included cybercrime in the national crime statistics for the first time in 2015, it doubled the crime rate overnight. Millions of people worldwide are victimised by online scams, whether it’s blocking access to a website, stealing personal or credit card information, or attempting to extort money by remotely holding the contents of a personal computer hostage.

“Since 2005, the police have largely ignored cybercrime,” says Professor Ross Anderson of Cambridge’s Computer Laboratory. “Reported crime fell by as much as a half in some categories. Yet, now that online and electronic fraud are included, the number of reported crimes has more than doubled. Crime was not falling; it was just moving online.”

In 2015, computer scientists, criminologists and legal academics joined forces to form the Cambridge Cybercrime Centre, with funding from the Engineering and Physical Sciences Research Council. Their aim is to help governments, businesses and ordinary users to construct better defences.

To understand how the criminals operate, researchers use machine learning and other techniques to recognise bad websites, understand what kinds of brands tend to be attacked and how often, determine how many criminals are behind an attack by looking at the pattern of the creation of fake sites and how effective the various defence systems are at getting them taken down.

One way in which studying cybercrime differs from many other areas of research is that the datasets are difficult to come by. Most belong to private companies, and researchers need to work hard to negotiate access. This is generally done through nondisclosure agreements, even if the data is out of date. And once researchers complete their work, they cannot make the data public, since it would reduce the competitive advantage of corporate players, and it may also make it possible for criminals to reverse engineer what was detected (and what wasn’t) and stay one step ahead of law enforcement.

One of the goals of the Cambridge Cybercrime Centre is to make it easier for cybercrime researchers from around the world to get access to data and share their results with colleagues.

To open up cybercrime research to colleagues across the globe, the team will leverage their existing relationships to collect and store cybercrime datasets, and then any bona fide researcher can sign a licence with the Centre and get to work without all the complexity of identifying and approaching the data holders themselves.

“Right now, getting access to data in this area is incredibly complicated,” says Dr Richard Clayton of Cambridge’s Computer Laboratory, who is also Director of the Centre. “But we think the framework we’ve set up will create a step change in the amount of work in cybercrime that uses real data. More people will be able to do research, and by allowing others to work on the same datasets more people will be able to do reproducible research and compare techniques, which is done extremely rarely at the moment.”

One of the team helping to make this work is Dr Julia Powles, a legal researcher cross-appointed between the Computer Laboratory and Faculty of Law. “There are several hurdles to data sharing,” says Powles. “Part of my job is to identify which ones are legitimate – for example, when there are genuine data protection and privacy concerns, or risks to commercial interests – and to work out when we are just dealing with paper tigers. We are striving to be as clear, principled and creative as possible in ratcheting up research in this essential field.”

Better research will make for better defences for governments, businesses and ordinary users. Today, there are a lot more tools to help users defend themselves against cybercrime – browsers are getting better at recognising bad URLs, for example – but, at the same time, criminals are becoming ever more effective, and more and more people are getting caught in their traps.

“You don’t actually have to be as clever as people once thought in order to fool a user,” says Clayton when explaining how fake bank websites are used to ‘phish’ for user credentials. “It used to be that cybercriminals would register a new domain name, like Barclays with two Ls, for instance. But they generally don’t do that for phishing attacks anymore, as end users aren’t looking at the address bar, they’re looking at whether the page looks right, whether the logos look right.”

The Centre is also looking at issues around what motivates someone to commit cybercrime, and what makes them stop.

According to Dr Alice Hutchings, a criminologist specialising in cybercrime, cybercriminals tend to fall into two main categories. The first category is the opportunistic offender, who may be motivated by a major strain in their lives, such as financial pressures or problems with gambling or addiction, and who uses cybercrime as a way to meet their goals. The second type of offender typically comes from a more stable background, and is gradually exposed to techniques for committing cybercrime through associations with others.

Both groups will usually keep offending as long as cybercrime meets their particular needs, whether it’s financial gratification, or supporting a drug habit, or giving them recognition within their community. What often makes offenders stop is the point at which the costs of continuing outweigh the benefits: for instance, when it takes a toll on their employment, other outside interests or personal relationships.

“Most offenders never get caught, so there’s no reason to think that they won’t go back to cybercrime,” says Hutchings. “They can always start again if circumstances in their lives change.

“There is so much cybercrime happening out there. You can educate potential victims, but there will always be other potential victims, and new ways that criminals can come up with to social engineer somebody’s details, for example. Proactive prevention against potential offenders is a good place to start.”

Criminologist Professor Lawrence Sherman believes the collaboration between security engineering and criminology is long overdue, both at Cambridge and globally: “Cybercrime is the crime of this century, a challenge we are just beginning to understand and challenge with science.”

“We’re extremely grateful to the people giving us this data, who are doing it because they think academic research will make a difference,” says Clayton.  “Our key contribution is realising that there was a roadblock in terms of being able to distribute the data. It’s not that other people couldn’t get the data before, but it was very time-consuming, so only a limited number of people were doing research in this area – we want to change that.”

“Our Cybercrime Centre will not only provide detailed technical information about what’s going on, so that firms can construct better defences,” says Anderson. “It will also provide strategic information, as a basis for making better policy.”

As more and more crime moves online, computer scientists, criminologists and legal academics have joined forces in Cambridge to improve our understanding and responses to cybercrime, helping governments, businesses and ordinary users construct better defences.

You don’t actually have to be as clever as people once thought in order to fool a user
Richard Clayton
TeQi's Graffitti Phish

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Artificial intelligence: computer says YES (but is it right?)

By lw355 from University of Cambridge - digital society. Published on Oct 20, 2016.

There would always be a first death in a driverless car and it happened in May 2016. Joshua Brown had engaged the autopilot system in his Tesla when a tractor-trailor drove across the road in front of him. It seems that neither he nor the sensors in the autopilot noticed the white-sided truck against a brightly lit sky, with tragic results.

Of course many people die in car crashes every day – in the USA there is one fatality every 94 million miles, and according to Tesla this was the first known fatality in over 130 million miles of driving with activated autopilot. In fact, given that most road fatalities are the result of human error, it has been said that autonomous cars should make travelling safer.

Even so, the tragedy raised a pertinent question: how much do we understand – and trust – the computers in an autonomous vehicle? Or, in fact, in any machine that has been taught to carry out an activity that a human would do?

We are now in the era of machine learning. Machines can be trained to recognise certain patterns in their environment and to respond appropriately. It happens every time your digital camera detects a face and throws a box around it to focus, or the personal assistant on your smartphone answers a question, or the adverts match your interests when you search online.

Machine learning is a way to program computers to learn from experience and improve their performance in a way that resembles how humans and animals learn tasks. As machine learning techniques become more common in everything from finance to healthcare, the issue of trust is becoming increasingly important, says Zoubin Ghahramani, Professor of Information Engineering in Cambridge's Department of Engineering.

Faced with a life or death decision, would a driverless car decide to hit pedestrians, or avoid them and risk the lives of its occupants? Providing a medical diagnosis, could a machine be wildly inaccurate because it has based its opinion on a too-small sample size? In making financial transactions, should a computer explain how robust is its assessment of the volatility of the stock markets?

“Machines can now achieve near-human abilities at many cognitive tasks even if confronted with a situation they have never seen before, or an incomplete set of data,” says Ghahramani. “But what is going on inside the ‘black box’? If the processes by which decisions were being made were more transparent, then trust would be less of an issue.”

His team builds the algorithms that lie at the heart of these technologies (the “invisible bit” as he refers to it). Trust and transparency are important themes in their work: “We really view the whole mathematics of machine learning as sitting inside a framework of understanding uncertainty. Before you see data – whether you are a baby learning a language or a scientist analysing some data – you start with a lot of uncertainty and then as you have more and more data you have more and more certainty.

“When machines make decisions, we want them to be clear on what stage they have reached in this process. And when they are unsure, we want them to tell us.”

One method is to build in an internal self-evaluation or calibration stage so that the machine can test its own certainty, and report back.

Two years ago, Ghahramani’s group launched the Automatic Statistician with funding from Google. The tool helps scientists analyse datasets for statistically significant patterns and, crucially, it also provides a report to explain how sure it is about its predictions.

“The difficulty with machine learning systems is you don’t really know what’s going on inside – and the answers they provide are not contextualised, like a human would do. The Automatic Statistician explains what it’s doing, in a human-understandable form.”

Where transparency becomes especially relevant is in applications like medical diagnoses, where understanding the provenance of how a decision is made is necessary to trust it.

Dr Adrian Weller, who works with Ghahramani, highlights the difficulty: “A particular issue with new artificial intelligence (AI) systems that learn or evolve is that their processes do not clearly map to rational decision-making pathways that are easy for humans to understand.” His research aims both at making these pathways more transparent, sometimes through visualisation, and at looking at what happens when systems are used in real-world scenarios that extend beyond their training environments – an increasingly common occurrence.

“We would like AI systems to monitor their situation dynamically, detect whether there has been a change in their environment and – if they can no longer work reliably – then provide an alert and perhaps shift to a safety mode.” A driverless car, for instance, might decide that a foggy night in heavy traffic requires a human driver to take control.

Weller’s theme of trust and transparency forms just one of the projects at the newly launched £10 million Leverhulme Centre for the Future of Intelligence (CFI). Ghahramani, who is Deputy Director of the Centre, explains: “It’s important to understand how developing technologies can help rather than replace humans. Over the coming years, philosophers, social scientists, cognitive scientists and computer scientists will help guide the future of the technology and study its implications – both the concerns and the benefits to society.”

CFI brings together four of the world’s leading universities (Cambridge, Oxford, Berkeley and Imperial College, London) to explore the implications of AI for human civilisation. Together, an interdisciplinary community of researchers will work closely with policy-makers and industry investigating topics such as the regulation of autonomous weaponry, and the implications of AI for democracy.

Ghahramani describes the excitement felt across the machine learning field: “It’s exploding in importance. It used to be an area of research that was very academic – but in the past five years people have realised these methods are incredibly useful across a wide range of societally important areas.

“We are awash with data, we have increasing computing power and we will see more and more applications that make predictions in real time. And as we see an escalation in what machines can do, they will challenge our notions of intelligence and make it all the more important that we have the means to trust what they tell us.”

Artificial intelligence has the power to eradicate poverty and disease or hasten the end of human civilisation as we know it – according to a speech delivered by Professor Stephen Hawking 19 October 2016 at the launch of the Centre for the Future of Intelligence.

Computers that learn for themselves are with us now. As they become more common in ‘high-stakes’ applications like robotic surgery, terrorism detection and driverless cars, researchers ask what can be done to make sure we can  trust them.

As we see an escalation in what machines can do, they will challenge our notions of intelligence and make it all the more important that we have the means to trust what they tell us
Zoubin Ghahramani
2019 by ExperiensS

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Algorithm for predicting protein pairings could help show how living systems work

By sc604 from University of Cambridge - Big data. Published on Sep 20, 2016.

Researchers have developed an algorithm that aids our understanding of how living systems work, by identifying which proteins within cells will interact with each other, based on their genetic sequences alone.

The ability to generate huge amounts of data from genetic sequencing has developed rapidly in the past decade, but the trouble for researchers is in being able to apply that sequence data to better understand living systems. The new research, published in the journal Proceedings of the National Academy of Sciences, is a significant step forward because biological processes, such as how our bodies turn food into energy, are driven by specific protein-protein interactions.

“We were really surprised that our algorithm was powerful enough to make accurate predictions in the absence of experimentally-derived data,” said study co-author Dr Lucy Colwell, from the University of Cambridge’s Department of Chemistry, who led the study with Ned Wingreen of Princeton University. “Being able to predict these interactions will help us understand how proteins fit and work together to complete required tasks – and using an algorithm is much faster and much cheaper than relying on experiments.”

When proteins interact with each other, they stick together to form protein complexes. In her previous research, Colwell found that if the two interacting proteins were known, sequence data could be used to figure out the structure of these complexes. Once the structure of the complexes is known, researchers can then investigate what is happening chemically. However, the question of which proteins interact with each other still required expensive, time-consuming experiments. Each cell often contains multiple versions of the same protein, and it wasn’t possible to predict which version of each protein would interact specifically – instead, experiments involve trying all options to see which ones stick.

In the current paper, the researchers used a mathematical algorithm to sift through the possible interaction partners and identify pairs of proteins that interact with each other. The method correctly predicted 93% of protein-protein interactions present in a dataset of more than 40,000 protein sequences for which the pairing is known, without being first provided any examples of correct pairs.

When two proteins stick together, some amino acids on one chain stick to the amino acids on the other chain. The boundaries between interacting proteins tend to evolve together over time, causing their sequences to mirror each other.

The algorithm uses this effect to build a model of the interaction. It first randomly pairs protein versions within each organism – because interacting pairs tend to be more similar in sequence to one another than non-interacting pairs, the algorithm can quickly identify a small set of largely correct pairings from the random starting point.

Using this small set, the algorithm measures whether the amino acid at a particular location in the first protein influences which amino acid occurs at a particular location in the second protein. These dependencies, learned from the data, are incorporated into a model and used to calculate the interaction strengths for each possible protein pair. Low-scoring pairings are eliminated, and the remaining set used to build an updated model.

The researchers thought that the algorithm would only work accurately if it first ‘learned’ what makes a good protein-protein pair by studying pairs that have been discovered in experiments. This meant that the researchers had to give the algorithm some known protein pairs, or ‘gold standards,’ against which to compare new sequences. The team used two well-studied families of proteins, histidine kinases and response regulators, which interact as part of a signaling system in bacteria.

But known examples are often scarce, and there are tens of millions of undiscovered protein-protein interactions in cells. So the team decided to see if they could reduce the amount of training they gave the algorithm. They gradually lowered the number of known histidine kinase-response regulator pairs that they fed into the algorithm, and were surprised to find that the algorithm continued to work. Finally, they ran the algorithm without giving it any such training pairs, and it still predicted new pairs with 93 percent accuracy.

“The fact that we didn't need a set of training data was really surprising,” said Colwell.

The algorithm was developed using proteins from bacteria, and the researchers are now extending the technique to other organisms. “Reactions in living organisms are driven by specific protein interactions,” said Colwell. “This approach allows us to identify and probe these interactions, an essential step towards building a picture of how living systems work.”

The research was supported in part by the National Institutes of Health, the National Science Foundation and the European Union.

Reference:
Anne-Florence Bitbol et al. ‘Inferring interaction partners from protein sequences.’ Proceedings of the National Academy of Sciences (2016). DOI: 10.1073/pnas.1606762113

An algorithm which models how proteins inside cells interact with each other will enhance the study of biology, and sheds light on how proteins work together to complete tasks such as turning food into energy.

Being able to predict these interactions will help us understand how proteins fit and work together to complete required tasks.
Lucy Colwell
Interacting proteins

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Cambridge to research future computing tech that could “ignite a technology field”

By tdk25 from University of Cambridge - Big data. Published on Apr 15, 2016.

A project which aims to establish the UK as an international leader in the development of “superconducting spintronics” – technology that could significantly increase the energy-efficiency of data centres and high-performance computing – has been announced.

Led by researchers at the University of Cambridge, the “Superspin” project aims to develop prototype devices that will pave the way for a new generation of ultra-low power supercomputers, capable of processing vast amounts of data, but at a fraction of the huge energy consumption of comparable facilities at the moment.

As more economic and cultural activity moves online, the data centres which house the servers needed to handle internet traffic are consuming increasing amounts of energy. An estimated three per cent of power generated in Europe is, for example, already used by data centres, which act as repositories for billions of gigabytes of information.

Superconducting spintronics is a new field of scientific investigation that has only emerged in the last few years. Researchers now believe that it could offer a pathway to solving the energy demands posed by high performance computing.

As the name suggests, it combines superconducting materials – which can carry a current without losing energy as heat – with spintronic devices. These are devices which manipulate a feature of electrons known as their “spin”, and are capable of processing large amounts of information very quickly.

Given the energy-efficiency of superconductors, combining the two sounds like a natural marriage, but until recently it was also thought to be completely impossible. Most spintronic devices have magnetic elements, and this magnetism prevents superconductivity, and hence reduces any energy-efficiency benefits.

Stemming from the discovery of spin polarized supercurrents in 2010 at the University of Cambridge, recent research, along with that of other institutions, has however shown that it is possible to power spintronic devices with a superconductor. The aim of the new £2.7 million project, which is being funded by the Engineering and Physical Sciences Research Council, is to use this as the basis for a new style of computing architecture.

Although work is already underway in several other countries to exploit superconducting spintronics, the Superspin project is unprecedented in terms of its magnitude and scope.

Researchers will explore how the technology could be applied in future computing as a whole, examining fundamental problems such as spin generation and flow, and data storage, while also developing sample devices. According to the project proposal, the work has the potential to establish Britain as a leading centre for this type of research and “ignite a technology field.”

The project will be led by Professor Mark Blamire, Head of the Department of Materials Sciences at the University of Cambridge, and Dr Jason Robinson, University Lecturer in Materials Sciences, Fellow of St John’s College, University of Cambridge, and University Research Fellow of the Royal Society. They will work with partners in the University’s Cavendish Laboratory (Dr Andrew Ferguson) and at Royal Holloway, London (Professor Matthias Eschrig).

Blamire and Robinson’s core vision of the programme is “to generate a paradigm shift in spin electronics, using recent discoveries about how superconductors can be combined with magnetism.” The programme will provide a pathway to making dramatic improvements in computing energy efficiency.

Robinson added: “Many research groups have recognised that superconducting spintronics offer extraordinary potential because they combine the properties of two traditionally incompatible fields to enable ultra-low power digital electronics.”

“However, at the moment, research programmes around the world are individually studying fascinating basic phenomena, rather than looking at developing an overall understanding of what could actually be delivered if all of this was joined up. Our project will aim to establish a closer collaboration between the people doing the basic science, while also developing demonstrator devices that can turn superconducting spintronics into a reality.”

The initial stages of the five-year project will be exploratory, examining different ways in which spin can be transported and magnetism controlled in a superconducting state. By 2021, however, the team hope that they will have manufactured sample logic and memory devices – the basic components that would be needed to develop a new generation of low-energy computing technologies.

The project will also report to an advisory board, comprising representatives from several leading technology firms, to ensure an ongoing exchange between the researchers and industry partners capable of taking its results further.

“The programme provides us with an opportunity to take international leadership of this as a technology, as well as in the basic science of studying and improving the interaction between superconductivity and magnetism,” Blamire said. “Once you have grasped the physics behind the operation of a sample device, scaling up from the sort of models that we are aiming to develop is not, in principle, too taxing.”

A Cambridge-led project aiming to develop a new architecture for future computing based on superconducting spintronics - technology designed to increase the energy-efficiency of high-performance computers and data storage - has been announced.

Superconducting spintronics offer extraordinary potential because they combine the properties of two traditionally incompatible fields to enable ultra-low power digital electronics
Jason Robinson
Growing quantities of data storage online are driving up the energy costs of high-performance computing and data centres. Superconducting spintronics offer a potential means of significantly increasing their energy-efficiency to resolve this problem.

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Graduate earnings: what you study and where matters – but so does parents’ income

By fpjl2 from University of Cambridge - Big data. Published on Apr 13, 2016.

Latest research has shown that graduates from richer family backgrounds earn significantly more after graduation than their poorer counterparts, even after completing the same degrees from the same universities.

The finding is one of many from a new study, published today, which looks at the link between earnings and students’ background, degree subject and university.

The research also found that those studying medicine and economics earn far more than those studying other degree subjects, and that there is considerable variation in graduates’ earnings depending on the university attended.

The study was carried out by the Institute of Fiscal Studies and the universities of Cambridge and Harvard, including Professor Anna Vignoles from Cambridge’s Faculty of Education. It is the first time a ‘big data’ approach has been used to look at how graduate earnings vary by institution of study, degree subject and parental income.

The researchers say that many other factors beyond graduate earnings, such as intrinsic interest, will and should drive student choice. However, they write that the research shows the potential value of providing some useful information that might inform students’ choice of degree – particularly to assist those from more disadvantaged backgrounds who might find it harder to navigate the higher education system.

“It would seem important to ensure there is adequate advice and guidance given that graduates’ future earnings are likely to vary depending on the institution and subject they choose, with implications for social mobility,” write the researchers in the study’s executive summary.

The research used anonymised tax data and student loan records for 260,000 students up to ten years after graduation. The dataset includes cohorts of graduates who started university in the period 1998-2011 and whose earnings (or lack of earnings) are then observed over a number of tax years. The paper focuses on the tax year 2012/13.

The study found that those from richer backgrounds (defined as being approximately from the top 20% of households of those applying to higher education in terms of family income) did better in the labour market than the other 80% of students.

The average gap in earnings between students from higher and lower income backgrounds is £8,000 a year for men and £5,300 a year for women, ten years after graduation.

Even after taking account of subject studied and the characteristics of the institution of study, the average student from a higher income background earned about 10% more than other students.

The gap is bigger at the top of the distribution – the 10% highest earning male graduates from richer backgrounds earned about 20% more than the 10% highest earners from relatively poorer backgrounds. The equivalent premium for the 10% highest earning female graduates from richer backgrounds was 14%.

The study also showed that graduates are much more likely to be in work, and earn much more than non-graduates. Non-graduates are twice as likely to have no earnings as are graduates ten years on (30% against 15% for the cohort who enrolled in higher education in 1999).

Partly as a result of this, half of non-graduate women had earnings below £8,000 a year at around age 30, say the researchers. Only a quarter of female graduates were earning less than this. Half were earning more than £21,000 a year.

Among those with significant earnings (which the researchers define as above £8,000 a year), median earnings for male graduates ten years after graduation were £30,000. For non-graduates of the same age median earnings were £21,000. The equivalent figures for women with significant earnings were £27,000 and £18,000.

“The research illustrates strongly that, for most graduates, higher education leads to much better earnings than those earned by non-graduates, although students need to realise that their subject choice is important in determining how much of an earnings advantage they will have,” said Professor Vignoles.

The researchers also found substantial differences in earnings according to which university was attended, as well as which subject was studied. They say however that this is in large part driven by differences in entry requirements.  

For instance, more than 10% of male graduates from LSE, Oxford and Cambridge were earning in excess of £100,000 a year ten years after graduation, with LSE graduates earning the most. LSE was the only institution with more than 10% of its female graduates earning in excess of £100,000 a year ten years on.

Even without focusing on the very top, the researchers say they found a large number of institutions (36 for men and 10 for women) had 10% of their graduates earning more than £60,000 a year ten years on. At the other end of the spectrum, there were some institutions (23 for men and 9 for women) where the median graduate earnings were less than those of the median non-graduate ten years on.

However, the researchers say that it is important to put this in context. “Given regional differences in average wages, some very locally focused institutions may struggle to produce graduates whose wages outpace English wide earnings, which includes those living in London where full time earnings for males are around 50% higher than in some other regions, such as Northern Ireland,” they write.

In terms of earnings according to subject, medical students were easily the highest earners at the median ten years out, followed by those who studied economics. For men, median earnings for medical graduates were about £50,000 after ten years, and for economics graduates £40,000.

Those studying the creative arts had the lowest earnings, and earned no more on average than non-graduates. However, the researchers say that some of these earnings differences are, of course, attributable to differences in student intake – since students with different levels of prior achievement at A-level take different subject options.

“When we account for different student intakes across subjects, only economics and medicine remain outliers with much higher earnings at the median as compared to their peers in other subjects,” write the researchers.

After allowing for differences in the characteristics of those who take different subjects, male medical graduates earn around £13,000 more at the median than similar engineering and technology graduates, the gap for women is approximately £16,000. Both male and female medical graduates earn around £14,000 more at the median than similar law graduates.

“Earnings vary substantially with university, subject, gender and cohort,” said study co-author Neil Shepherd of Harvard University. “This impacts on which parts of the HE sector the UK Government funds through the subsidy inherent within income contingent student loans. The next step in the research is to quantifying that variation in funding, building on today's paper.”

Reference:
Institute for Fiscal Studies working paper: 'How English domiciled graduate earnings vary with gender, institution attended, subject and socio-economic background', Jack Britton , Lorraine Dearden , Neil Shephard and Anna Vignoles.

First ‘big data’ research approach to graduate earnings reveals significant variations depending on student background, degree subject and university attended.  

The research illustrates strongly that, for most graduates, higher education leads to much better earnings than those earned by non-graduates, although students need to realise that their subject choice is important in determining how much of an earnings advantage they will have
Anna Vignoles
Sidney Sussex General Admission, Cambridge 2012

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Predicting gentrification through social networking data

By sc604 from University of Cambridge - Big data. Published on Apr 13, 2016.

The first network to look at the interconnected nature of people and places in large cities is not only able to quantify the social diversity of a particular place, but can also be used to predict when a neighbourhood will go through the process of gentrification, which is associated with the displacement of residents of a deprived area by an influx of a more affluent population.

The researchers behind the study, led by the University of Cambridge, will present their results today (13 April) at the 25th International World Wide Web Conference in Montréal.

The Cambridge researchers, working with colleagues from the University of Birmingham, Queen Mary University of London, and University College London, used data from approximately 37,000 users and 42,000 venues in London to build a network of Foursquare places and the parallel Twitter social network of visitors, adding up to more than half a million check-ins over a ten-month period. From this data, they were able to quantify the ‘social diversity’ of various neighbourhoods and venues by distinguishing between places that bring together strangers versus those that tend to bring together friends, as well as places that attract diverse individuals as opposed to those which attract regulars.

When these social diversity metrics were correlated with wellbeing indicators for various London neighbourhoods, the researchers discovered that signs of gentrification, such as rising housing prices and lower crime rates, were the strongest in deprived areas with high social diversity. These areas had an influx of more affluent and diverse visitors, represented by social media users, and pointed to an overall improvement of their rank, according to the UK Index of Multiple Deprivation.

The UK Index of Multiple Deprivation (IMD) is a statistical exercise conducted by the Department of Communities and Local Government, which measures the relative prosperity of neighbourhoods across England. The researchers compared IMD data for 2010, the year their social and place network data was gathered, with the IMD data for 2015, the most recent report.

“We’re looking at the social roles and properties of places,” said Desislava Hristova from the University’s Computer Laboratory, and the study’s lead author. “We found that the most socially cohesive and homogenous areas tend to be either very wealthy or very poor, but neighbourhoods with both high social diversity and high deprivation are the ones which are currently undergoing processes of gentrification.”

This aligns with previous research, which has found that tightly-knit communities are more resistant to changes and resources remain within the community. This suggests that affluent communities remain affluent and poor communities remain poor because they are relatively isolated.

Hristova and her co-authors found that of the 32 London boroughs, the borough of Hackney had the highest social diversity, and in 2010, had the second-highest deprivation. By 2015, it had also seen the most improvement on the IMD index, and is now an area undergoing intense gentrification, with house prices rising far above the London average, fast-decreasing crime rate and a highly diverse population.

In addition to Hackney, Tower Hamlets, Greenwich, Hammersmith and Lambeth are also boroughs with high social diversity and high deprivation in 2010, and are now undergoing the process of gentrification, with all of the positive and negative effects that come along with it.

The ability to predict the gentrification of neighbourhoods could help local governments and policy-makers improve urban development plans and alleviate the negative effects of gentrification while benefitting from economic growth.

In order to measure the social diversity of a given place or neighbourhood, the researchers defined four distinct measures: brokerage, serendipity, entropy and homogeneity. Brokerage is the ability of a place to connect people who are otherwise disconnected; serendipity is the extent to which a place can induce chance encounters between its visitors; entropy is the extent to which a place is diverse with respect to visits; and homogeneity is the extent to which the visitors to a place are homogenous in their characteristics.

Within categories of places, the researchers found that some places were more likely places for friends to meet, and some were for more fleeting encounters. For example, in the food category, strangers were more likely to meet at a dumpling restaurant while friends were more likely to meet at a fried chicken restaurant. Similarly, friends were more likely to meet at a B&B, football match or strip club, while strangers were more likely to meet at a motel, art museum or gay bar.

“We understand that people who diversify their contacts socially and geographically have high social capital, but what about places?” said Hristova. “We all have a general notion of the social diversity of places and the people that visit them, but we’ve attempted to formalise this – it could even be used as a specialised local search engine.”

For instance, while there are a number of ways a tourist can find a highly-recommended restaurant in a new city, the social role that a place plays in a city is normally only known by locals through experience. “Whether a place is touristy or quiet, artsy or mainstream could be integrated into mobile system design to help newcomers or tourists feel like locals,” said Hristova.

Reference:
Desislava Hristova et al. ‘Measuring Urban Social Diversity Using Interconnected Geo-Social Networks.’ Paper presented to the International World Wide Web Conference, Montréal, 11-15 April 2016. http://www2016.ca/program-at-a-glance.html

Data from location-based social networks may be able to predict when a neighbourhood will go through the process of gentrification, by identifying areas with high social diversity and high deprivation.

We understand that people who diversify their contacts socially and geographically have high social capital, but what about places?
Desislava Hristova
Gentrification in Progress

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes
License type: 

Opinion: Here’s how tweets and check-ins can be used to spot early signs of gentrification

By Anonymous from University of Cambridge - Big data. Published on Apr 12, 2016.

When you walk through a neighbourhood undergoing gentrification, you can sense it – the area is dominated by strange contradictions. Public spaces are populated by vagabonds and cool kids; abandoned buildings sit in disrepair next to trendy coffee shops; blocks of council housing abut glassy new developments.

Urbanists describe gentrification as a form of urban migration, where a more affluent population displaces the original, lower-income population. In statistics, gentrification appears as the lowering of crime rates, rising housing prices and changes to the mix of people who live there.

If we could only predict where gentrification is likely to strike next, we might be able to alleviate its negative impacts – such as displacement – and take advantage of its more positive effects, which include economic growth. That’s why our latest study – conducted with colleagues at the University of Birmingham, Queen Mary University of London, and University College London – aimed to quantify the process of gentrification, and discover the warning signs.

Detecting urban diversity

We constructed four measures of urban social diversity using data from social media. By combining these measures with government statistics about deprivation, we were able to pinpoint a number of neighbourhoods undergoing gentrification in London.

Of course, social media is notoriously unsuitable for population studies, because of the “digital divide”: the split between people who can access the internet and those who can’t exists even within urban areas – so information from social media only captures part of the overall picture. Twitter users in particular are known to be predominantly young, affluent and living in urban areas.

But these are precisely the demographics responsible for gentrification. So, we used information from social media from 2010 and 2011 to define the “social diversity” of urban venues such as restaurants, bars, schools and parks.

Urban social diversity – in terms of population, economy and architecture – is known to be a factor in successful communities. In her famous book The Death and Life of Great American Cities, urban activist Jane Jacobs wrote that “cities differ from towns and suburbs in basic ways, and one of these is that cities are, by definition, full of strangers”.

Dropping in. David Abrahamovitch/Flickr, CC BY

In our work, we first measured the amount of strangers that a place brings together as the fraction of the social network of visitors who are connected on social media. This gave us an idea of whether a place tends to be frequented by strangers or friends. We further explored the diversity of these visitors in terms of their mobility preferences and spontaneity in choice of venues. Although we did not consider demographics or income levels, there is a known relationship between the wealth of people and the diversity of their geographical interactions.

We studied the social network of 37,000 London users of Twitter, and combined it with what we knew about their mobility patterns from geo-located Foursquare check-ins posted to their public profiles.

By studying the amount of strangers versus friends meeting at a bar, or the number of diverse versus similar individuals visiting an art gallery, we were able to quantify the overall diversity of London neighbourhoods, in terms of their visitors.

Networks are powerful representations of the relationships between people and places. Not only can we draw links between people where a relationship – such as friendship – exists between them; we can also draw connections between two places if a visitor has been to both. We can even connect the two networks, by drawing links between people in the social network who have visited specific spots in the place network.

In this way, we are able to extract the social network of a place, and the place network of a person. By the time we’d finished crunching the data, we could take stock of the range of people who had visited a specific place, and the different places visited by any individual.

When we compared the diversity of urban neighbourhoods with official government statistics on deprivation, we found that some highly deprived areas were also extremely socially diverse. In other words, there were lots of diverse social media users visiting some of London’s poorest neighbourhoods.

Diminishing deprivation

To find out what was going on, we took the newly published deprivation indices for 2015 and looked for changes in the levels of deprivation from our study period in 2011. The relationship was striking. The areas where we saw high levels of social diversity and extreme deprivation in 2011, were exactly the same areas that had experienced the greatest decreases in deprivation by 2016.

A prime example can be found in the London borough of Hackney. Anyone visiting Hackney might describe it in terms of the contradictions we mentioned before – but few of us could afford to live there today. In our study, Hackney was the highest ranking in deprivation and the highest ranking in social diversity in 2011. Between then and now, it has gone from the being the second most deprived neighbourhood in the country, to the 11th.

So, although social media may not be representative of the entire population, it can offer the key to measuring and understanding the processes of gentrification. Neither entirely good nor thoroughly bad, gentrification is a phenomenon that we should all watch out for. It will undoubtedly help to define how our cities transform in years to come.

Desislava Hristova, PhD Candidate, University of Cambridge

This article was originally published on The Conversation. Read the original article.

The opinions expressed in this article are those of the individual author(s) and do not represent the views of the University of Cambridge.

Desislava Hristova (Computer Laboratory) discusses how data from location-based social networks can be used to predict when a neighbourhood will go through the process of gentrification.

Creative Commons License
The text in this work is licensed under a Creative Commons Attribution 4.0 International License. For image use please see separate credits above.

Yes

Alan Turing Institute Doctoral Fellowships now open for October 2016 entry

From Cambridge Big Data. Published on Jan 25, 2016.

New institute is set to push the boundaries of information science

From Cambridge Big Data. Published on Nov 12, 2015.

Philanthropic support of £5m from Cantab Capital Partners, establishes the Cantab Capital Institute for the Mathematics of Information.

Big Data for Small and Medium Enterprises - register now for Alan Turing Institute Summit

From Cambridge Big Data. Published on Sep 07, 2015.

Join representatives of academia and industry on 16th November 2015 to identify key challenges in data science for small and medium-sized enterprises.

Ethics of Big Data Research Group launched

From Cambridge Big Data. Published on Aug 12, 2015.

A new interdisciplinary research group at CRASSH will investigate the Ethics of Big Data during the coming academic year.

Cambridge to host Alan Turing Institute Summit on Data Science for Health

From Cambridge Big Data. Published on Aug 11, 2015.

Join researchers, practitioners and policymakers on September 21st to discuss priorities for applying Data Science to healthcare

Cambridge Public Policy publishes report on the Internet of Things

From Cambridge Big Data. Published on Aug 05, 2015.

The Internet of Things will not flourish without laws to protect privacy rights and civil dialogue to build trust in new technologies, says a new Report from the University of Cambridge

Collaborative workshops fund - closing date 1 September 2015

From Cambridge Big Data. Published on Jul 28, 2015.

CBD Travel Grants

From Cambridge Big Data. Published on Jul 28, 2015.

Professor Zoubin Ghahramani elected FRS

From Cambridge Big Data. Published on May 06, 2015.

Call for Workshop Proposals - closing date 31 May 2015

From Cambridge Big Data. Published on Apr 28, 2015.

Big Data and Policy: report from Cambridge workshop

From Cambridge Big Data. Published on Dec 22, 2014.

About us

The Cambridge Big Data Strategic Research Initiative brings together researchers from across the University to address challenges presented by our access to unprecedented volumes of data. Our research spans all six Schools of the University, from the underlying fundamentals in mathematics and computer science, to applications ranging from astronomy and bioinformatics, to medicine, social science and the humanities

In parallel, our research addresses important issues around law, ethics and economics, in order to apply Big Data to solve challenging problems for society.

Cambridge Big Data supports collaboration and knowledge transfer in this growing field. 

Click here to Join Us.