Skip to main content

The Problem of Dark Data

By
A March New York Times article sounded warning bells for researchers: the scourge of dark data. Dark data doesn’t refer to anything secret or illegal, but rather data developed by the government and other organizations subject to loss. A more complete definition, often used in the corporate context, is "the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes.” Concern over the loss of data that could lead to new discoveries has been especially equated with the loss of scientific data stored by agencies and other organizations. Much of this data is stored on government servers, with no legal obligation to remain available. The Trump administration’s proposed cuts to scientific research and agency funding has only increased the alarm felt by scientists and other researchers.

An additional problem is that dark data, by definition, is unknown. It can’t be verified if it can’t be found, even though we know it’s there. Somewhere. Right now, data.gov is the central repository for government created databases, but it relies on agencies to self-report and is, by many researchers’ estimates, only a fraction of data created by the agencies. The use of proprietary code and data.gov’s practice of linking to data housed on websites, instead of the databases themselves, makes it even more difficult for researchers.

While there does not seem to be any federal legislation prohibiting the destruction or decentralization of these types of data, several non-profits have formed to save this data from going dark, by identifying and downloading  data viewed as vulnerable to deletion.

To learn more about dark data, here are some resources to get you started:

Politics: Turbulence Ahead (Nature)

A Decade of Discovery (podcast)

Use it or lose it—the search for enlightenment in dark data (Phys.org)

Dark Web: Exploring and Data Mining the Dark Side of the Web, Hsinchun Chen

Comments

Post a Comment

Popular posts from this blog

The Amazing, but True, Deportation Story of Carlos Marcello

By
Earlier this week, the University of Houston Law Center was fortunate to have as its guest Professor Daniel Kanstroom of Boston College of Law. An expert in immigration law, he is the Director of the International Human Rights Program, and he both founded and directs the Boston College Immigration and Asylum Clinic. Speaking as the guest of the Houston Journal of International Law’s annual Fall Lecture Series, Professor Kanstroom discussed issues raised in his new book, Aftermath: Deportation Law and the New American Diaspora . Professor Michael Olivas introduced Professor Kanstroom to the audience, and mentioned the fascinating tale of Carlos Marcello, which Professor Kanstroom wrote about in his chapter “The Long, Complex, and Futile Deportation Saga of Carlos Marcello,” in Immigration Stories , a collection of narratives about leading immigration law cases. My interest piqued, I read and was amazed by Kanstroom’s description of one of the most interesting figures in American le...
1 comment
Read more

This Day in Legal History -- Roy Cohn

By
Big lawyers have big personalities, and few had as big a personality as Roy Cohn who was born on this day in New York City in 1927. Cohn’s career was spent in the Washington DC—New York axis, famous in the press for his legal and social exploits, his list of famous and infamous clients, and as the poster-boy for every stereotype of the dishonest grasping lawyer. The legal career of Roy Cohn is an object lesson on the use, and abuse, of the lawyer’s power. Roy Cohn was born on February 20, 1927 in New York City. His father was a justice in the Appellate Division of the State Supreme Court and was well connected politically. These connections, along with a precocious intelligence which allowed him to graduate from Columbia law school at age 20, paved the way for a bright legal career. Cohn’s first job (after waiting until he turned 21 to be admitted to the bar) was as an assistant U.S. Attorney in New York. He first came to prominence in his role as a prosecutor in the Rosenberg espi...
Post a Comment
Read more

Lessons for Today from the Genocide Against the Tutsi in Rwanda

By Anonymous
“Man’s inhumanity to man is not only perpetrated by the vitriolic actions of those who are bad. It is also perpetrated by the vitiating inaction of those who are good.” –Martin Luther King Jr.   Last week, I had the pleasure of attending  Professor Zachary D. Kaufman ’s presentation on  Lessons for Today from the Genocide Against the Tutsi in Rwanda  hosted by the  Johannesburg Holocaust & Geno cide Ce ntre . Among the many takeaways highlighted by Professor Kaufman and drawn from  Lessons from Rwanda: Post-Genocide Law and Policy   were ten simple yet profound lessons:   Lesson #1: Hate speech is dangerous.   To illustrate the role that hate speech played in the Rwandan genocide, Professor Kaufman discussed multiple forms of  propaganda , such as Kangura, Radio Rwanda, and RTLM “hate radio.”   He concludes that we must have limits, including with respect to social media, and further asserts that social media must do a better jo...
Post a Comment
Read more