Skip to main content

The Problem of Dark Data

By
A March New York Times article sounded warning bells for researchers: the scourge of dark data. Dark data doesn’t refer to anything secret or illegal, but rather data developed by the government and other organizations subject to loss. A more complete definition, often used in the corporate context, is "the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes.” Concern over the loss of data that could lead to new discoveries has been especially equated with the loss of scientific data stored by agencies and other organizations. Much of this data is stored on government servers, with no legal obligation to remain available. The Trump administration’s proposed cuts to scientific research and agency funding has only increased the alarm felt by scientists and other researchers.

An additional problem is that dark data, by definition, is unknown. It can’t be verified if it can’t be found, even though we know it’s there. Somewhere. Right now, data.gov is the central repository for government created databases, but it relies on agencies to self-report and is, by many researchers’ estimates, only a fraction of data created by the agencies. The use of proprietary code and data.gov’s practice of linking to data housed on websites, instead of the databases themselves, makes it even more difficult for researchers.

While there does not seem to be any federal legislation prohibiting the destruction or decentralization of these types of data, several non-profits have formed to save this data from going dark, by identifying and downloading  data viewed as vulnerable to deletion.

To learn more about dark data, here are some resources to get you started:

Politics: Turbulence Ahead (Nature)

A Decade of Discovery (podcast)

Use it or lose it—the search for enlightenment in dark data (Phys.org)

Dark Web: Exploring and Data Mining the Dark Side of the Web, Hsinchun Chen

Comments

Post a Comment

Popular posts from this blog

The Amazing, but True, Deportation Story of Carlos Marcello

By
Earlier this week, the University of Houston Law Center was fortunate to have as its guest Professor Daniel Kanstroom of Boston College of Law. An expert in immigration law, he is the Director of the International Human Rights Program, and he both founded and directs the Boston College Immigration and Asylum Clinic. Speaking as the guest of the Houston Journal of International Law’s annual Fall Lecture Series, Professor Kanstroom discussed issues raised in his new book, Aftermath: Deportation Law and the New American Diaspora . Professor Michael Olivas introduced Professor Kanstroom to the audience, and mentioned the fascinating tale of Carlos Marcello, which Professor Kanstroom wrote about in his chapter “The Long, Complex, and Futile Deportation Saga of Carlos Marcello,” in Immigration Stories , a collection of narratives about leading immigration law cases. My interest piqued, I read and was amazed by Kanstroom’s description of one of the most interesting figures in American le...
1 comment
Read more

This Day in Legal History -- Roy Cohn

By
Big lawyers have big personalities, and few had as big a personality as Roy Cohn who was born on this day in New York City in 1927. Cohn’s career was spent in the Washington DC—New York axis, famous in the press for his legal and social exploits, his list of famous and infamous clients, and as the poster-boy for every stereotype of the dishonest grasping lawyer. The legal career of Roy Cohn is an object lesson on the use, and abuse, of the lawyer’s power. Roy Cohn was born on February 20, 1927 in New York City. His father was a justice in the Appellate Division of the State Supreme Court and was well connected politically. These connections, along with a precocious intelligence which allowed him to graduate from Columbia law school at age 20, paved the way for a bright legal career. Cohn’s first job (after waiting until he turned 21 to be admitted to the bar) was as an assistant U.S. Attorney in New York. He first came to prominence in his role as a prosecutor in the Rosenberg espi...
Post a Comment
Read more

The Bar Exam: Some History & Comparison

By Lauren Schroeder
It's that time of year again...time for two to three days of squeezing out all the material that hopeful lawyers have been pounding into their heads over several months - the bar exam. To mark the occasion, here is a brief history of the exam itself, as well as a taste of what is required to become an attorney elsewhere. Prior to the mid-1800s, there were no written bar exams. Instead, the path to becoming a lawyer led hopefuls through "apprenticeships, self-directed reading, and oral examinations." The next phase made use of a diploma privilege, which remained until the ABA began requiring exams in the 1920s. (A diploma privilege does still exist in Wisconsin, however.) The first state to employ a written version of the bar exam was Massachusetts, in 1855. (See Riebe, A Bar Review for Law Schools: Getting Students on Board to Pass Their Bar Exams , 45 Brandeis L. J. 269 (2007) for quoted material and historical information.) Bar exams consisted only of essays until...
Post a Comment
Read more