Data Science for NASA Open Data

Last modified

Story

Data Science for NASA Open Data

In preparation for the June 25th Meetup with Beth Beck, I need to understand the following:

I also know NASA CTO for IT Deborah Diaz from my previous work for the Federal CIO Council.

I have done Data Science for Data Science Central Meteorite Data previously and am interested to see if this is the same or different data.

A key challenge is to find the data and document how you found it, usually with a bit of luck, as was the case here.

I imported 4 Excel or CSV meteorite data sets (See Attachments Below) into Spotfire and created a Dashboard with Guided Analysis See Below) with Screen Captures in the Slides below. I could do more of these to create a Data Science Data Publication and MOOC (Massive Open Online Course) for the Space Apps Challenge like I have done and am doing for other agencies and programs as part of the Federal Big Data Working Group Meetup. I am  currently working on the NASA Data Science Data Publication for QUDT in cooperation with Dr. Joan Aron and Dr. Ralph Hodgson for our June 29th Meetup. See Research Notes.

Next I need to explore the 34 Other Visualizations of Meteorite Data to see if I have missed anything and then work with the NASA NEX. I think I may have already worked with some of these big data as part of my work on Data Science for CODATA and Data Science for Climate Change.

MORE TO FOLLOW

What is a Guided Analysis?

There may be times when you want to create and share an analysis file with other people and have them perform their own analysis on it. You might want to set up the analysis file to load particular data and show certain visualizations, but also provide instructions for other people on how to use the document. There might be a specific order a person should go through the pages, and detailed instructions on what to look for and which filters are relevant to manipulate on each page. To aid in this, there are a number of things you can do in TIBCO Spotfire to set up a guided flow through your document.

When the recipients of your analysis file open it, they will be guided through the analysis as per your instructions, but be able to do their own filtering and look closer at any noteworthy aspects they find interesting. This allows you to set up a generic analysis covering a subject such as sales over the entire United States, but instruct the recipients to filter down to the specific information of interest.

Slides

Slides 

Slide 1 Data Science for NASA Open Data: Meteorite Data

BrandNiemann06232015Slide1.PNG

Slide 2 Data Science for NASA Open Data: MindTouch Knowledge base

BrandNiemann06232015Slide2.PNG

Slide 3 Data Science for NASA Open Data: Meteorite Update Data-Spotfire Cover Page

BrandNiemann06232015Slide3.PNG

Slide 4 Data Science for NASA Open Data: Meteorite Landings Data-Spotfire

BrandNiemann06232015Slide4.PNG

Slide 5 Data Science for NASA Open Data: Meteorite Landings 2 Data-Spotfire

BrandNiemann06232015Slide5.PNG

Slide 6 Data Science for NASA Open Data: Meteorite Strikes Data-Spotfire

BrandNiemann06232015Slide6.PNG

Spotfire Dashboard

For Internet Explorer Users and Those Wanting Full Screen Display Use: Web Player Get Spotfire for iPad App

Error: Embedded data could not be displayed. Use Google Chrome

Research Notes

NASA Data Science Data Publication for QUDT:

http://semanticommunity.info/Data_Sc...of_Things/QUDT

This is for our federal Big Data Working Group Meetup on June 29, that uses NASA Open Data:
http://www.meetup.com/Federal-Big-Da...nts/222263009/

All are welcome to participate in person or remotely.

Open Data Discussion with NASA

Source: http://www.meetup.com/Open-Data-Lead...nts/222876408/

Thursday, June 25, 2015
6:30 PM

The Center for Data Innovation
1101 K Street NW, Suite 610, Washington, DC (map)

We're very happy to have NASA's Beth Beck, open innovation program manager, present on NASA's open data initiatives!

Beth will discuss the Space Apps Challenge (https://2015.spaceappschallenge.org/about/) and other open data projects at the agency. The Space Apps Challenge is a yearly international, 48-hour hackathon that challenges teams to tackle problems across four categories with the help of open data: outer space, the earth, humans, and robotics. The 2015 Space Apps Challenge concluded in May 2015 and had an incredible 13,714 participants working on 949 projects across 133 locations. 

Agenda: 6:30 PM - Arrival and networking  

7:00 PM - Presentation 

7:30 PM - Discussion

8:00 PM - Conclusion

A friendly reminder: out of respect to our presenters, please arrive as closely to 6:30 as possible so we can start on time.

SpaceApps Challenge

Source: https://2015.spaceappschallenge.org

Source: https://2015.spaceappschallenge.org/about/

The International Space Apps Challenge is an international mass collaboration focused on space exploration that takes place over 48-hours in cities around the world. The event embraces collaborative problem solving with a goal of producing relevant open-source solutions to address global needs applicable to both life on Earth and life in space. This year we have over 25 challenges in four areas: Earth, Outer Space, Humans and Robotics. NASA is leading this global collaboration along with a number of government collaborators and over 100 local organizing teams across the globe.

INTERNATIONAL
The exploration of space is, by necessity, a unified international effort - and diversity of experience and perspective inevitably produces a better product. We are very intentional to make the Space Apps Challenge valuable to and accessible to the international community, and the experience continues to teach us numerous lessons about how to make it more truly international.

SPACE APPS
The Challenge exemplifies the principles of transparency, participation and collaboration by utilizing openly available data, supplied through NASA missions and technology, and the talent and skill of passionate volunteers from around the planet to advance space exploration and improve the quality of life on Earth.

CHALLENGE
The idea of a Challenge is so compelling because it acknowledges the fact that the world is facing serious challenges - and that we all have to work together to approach them. While there local hosts may offer prizes and the global award includes an opportunity to attend a NASA launch, the main challenge we focus on is enabling 48 hours of highly engaged collaboration- and discovering what we can create when that happens.

Projects by Challenge

Source: https://2015.spaceappschallenge.org/project/

This page indexes each project from the 2015 SpaceApps event.

Airburst Data Visualization

Meteor Data Visualization Infography

Source: https://2015.spaceappschallenge.org/...on-infography/

Data Visualization 

This project is solving the Airburst Data Visualization challenge.
Description
Data Visualization

PROJECT INFORMATION

License: Open Software License 3.0 (OSL-3.0)

Source Code/Project URL: https://app.box.com/s/msrhn3i0kl652o...4xu9g1wr0nzc33

ATTACH Airburst Data Visualization.zip

RESOURCES

Airburst Data Visualization 2015 Challenge

Source: https://2015.spaceappschallenge.org/...visualization/

Hashtags

#outerspace, #airburstvisual, #intermediate

Tags

Model, Platform, Data Visualization

Background

NASA’s Asteroid Grand Challenge is dedicated to locating asteroids that might pose a threat to human populations and then, once located, knowing what to do about them. Chelyabinsk caught the world by surprise. We want to stop this from happening again.

Location of these small, fast-moving NEOs requires space-based hardware, however the threat of these airburst events is still largely misunderstood. Indeed as human cities grow and become more urbanized, the threat of a serious disaster this century is far higher than in the 20th. Some estimates put the threat now equal to that of dying in a commercial airliner - and yet we now have the technology to do something about it.

The Chelyabinsk meteor that exploded in the atmosphere over Russia was an example of a small Near Earth Object (about 17m wide) that exploded in the upper atmosphere, releasing levels of energy of about half a megaton.

This event created a fireball thirty times the power of the sun, burning retinas and the skin of observers, blowing out the windows of over 3600 buildings and knocking people off their feet. At least 1210 people were treated for injuries.

Data from infrasound studies has revealed that these sorts of airbursts - known as ‘Bolide events’ are far more frequent than previously thought. Indeed over a twenty year interval between 1994 and 2013 there were 556 events (of which Chelyabinsk was the most powerful.) NASA’s Asteroid Grand Challenge team requires innovative and compelling ways to visualize the airburst phenomenon.

Challenge

Create a way of visualizing the threat of atmospheric airbursts using newly released data gathered in the Bolide study. Build on any open source solutions you find, or create your own.

Your solution should both explain what an airburst is and highlight the frequency using JPL’s Bolide study data.

Considerations

Your solution should adhere closely to the known data of the Bolide study, but bring this to life so anyone in the world can understand its implications. Ideally your solution should be built with web-tools that can be embedded into a Content Management System (CMS), as needed or shared on social media.

Sample Resources

(Participants do not have to use these resources, and NASA in no way endorses any particular entity listed).

Relevant Datasets

(Participants do not have to use these resources, and NASA in no way endorses any particular entity listed).

NASA's Data Portal

Source: https://data.nasa.gov/

Statistics

Source: https://data.nasa.gov/data

Categories Types Number
All All 9126
Aerospace All 207
Applied Science All 1
Earth Science All 8302
Management Operations All 261
Space Science All 226
Earth Science Tabular Data 0
Earth Science Charts 0
Earth Science Maps 0
Earth Science Calendars 0
Earth Science Filtered Views 0
Earth Science External Links 8302
Earth Science Files and Documents 0
Earth Science Forms 0
All Tabular Data 15
All Charts 0
All Maps 0
All Calendars 0
All Filtered Views 0
All External Links 9111
All Files and Documents 0
All Forms 0

 

Meteorite Landings

https://data.nasa.gov/Space-Science/...ings/gh4g-9sfh

This comprehensive data set from The Meteoritical Society contains information on all of the known meteorite landings. The Fusion Table is collected by Javier de la Torre and we've also provided an XLS file that consists of 34,513 meteorites and includes the following fields: place type_of_meteorite mass_g fell_found year database coordinate_1 coordinates_2 cartodb_id created_at updated_at year_date longitude latitude geojson **5/14/13 Please find an updated data set from The Meteoritical Society that includes more recent meteorites. Under NameType, 'valid' is for most meteorites and 'relict' are for objects that were once meteorites but are now highly altered by weathering on Earth. http://visualizing.org/datasets/meteorite-landings

Visualizing.org

Source: http://visualizing.org/

Meteorite Landings

Source: http://visualizing.org/datasets/meteorite-landings

Download: http://visualizing.org/sites/default..._05-14-13.xlsx

Source: The Meteoritical Society

Description: This comprehensive data set from The Meteoritical Society contains information on all of the known meteorite landings. The Fusion Table is collected by Javier de la Torre and we've also provided an XLS file that consists of 34,513 meteorites and includes the following fields:

  • place
  • type_of_meteorite
  • mass_g
  • fell_found
  • year
  • database
  • coordinate_1
  • coordinates_2
  • cartodb_id
  • created_at
  • updated_at
  • year_date
  • longitude
  • latitude
  • geojson

**5/14/13
Please find an updated data set from The Meteoritical Society that includes more recent meteorites. Under NameType, 'valid' is for most meteorites and 'relict' are for objects that were once meteorites but are now highly altered by weathering on Earth.

34 Other Visualizations

NASA NEX is a collaboration and analytical platform that combines state-of-the-art supercomputing, Earth system modeling, workflow management and NASA remote-sensing data. Through NEX, users can explore and analyze large Earth science data sets, run and share modeling algorithms, collaborate on new or existing projects and exchange workflows and results within and among other science communities.

Three NASA NEX data sets are now available to all via Amazon S3. One data set, the NEX downscaled climate simulations, provides high-resolution climate change projections for the 48 contiguous U.S. states. The second data set, provided by the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument on NASA's Terra and Aqua satellites, offers a global view of Earth's surface every 1 to 2 days. Finally, the Landsat data record from the U.S. Geological Survey provides the longest existing continuous space-based record of Earth's land.

Accessing NASA NEX Data

AWS is making the NASA NEX data available to the community free of charge. There are a variety of ways to access the data:

The data is hosted for free by AWS as part of the AWS Public Data Sets program.

Available NASA NEX Data Sets

Downscaled Climate Projections (NEX-DCP30)

The NASA Earth Exchange (NEX) Downscaled Climate Projections (NEX-DCP30) dataset is comprised of downscaled climate scenarios for the conterminous United States that are derived from the General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 5 (CMIP5) [Taylor et al. 2012] and across the four greenhouse gas emissions scenarios known as Representative Concentration Pathways (RCPs) [Meinshausen et al. 2011] developed for the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR5). The dataset includes downscaled projections from 33 models, as well as ensemble statistics calculated for each RCP from all model runs available. The purpose of these datasets is to provide a set of high resolution, bias-corrected climate change projections that can be used to evaluate climate change impacts on processes that are sensitive to finer-scale climate gradients and the effects of local topography on climate conditions.

Each of the climate projections includes monthly averaged maximum temperature, minimum temperature, and precipitation for the periods from 1950 through 2005 (Retrospective Run) and from 2006 to 2099 (Prospective Run).

Available at s3://nasanex/NEX-DCP30

Global Daily Downscaled Projections (NEX-GDDP)

The NASA Earth Exchange (NEX) Global Daily Downscaled Projections (NEX-GDDP) dataset is comprised of downscaled climate scenarios that are derived from the General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 5 (CMIP5) [Taylor et al. 2012] and across the two of the four greenhouse gas emissions scenarios known as Representative Concentration Pathways (RCPs) [Meinshausen et al. 2011] developed for the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR5). The dataset is an ensemble of projections from 21 different models and two RCPs (RCP 4.5 and RCP 8.5), and provides daily estimates of maximum and minimum temperatures and precipitation using a daily Bias-Correction - Spatial Disaggregation (BCSD) method (Thrasher, et al., 2012). The data spans the entire globe with a 0.25 degree (~25-kilometer) spatial resolution for the periods from 1950 through 2005 (Historical) and from 2006 to 2100 (Climate Projections).

Available at s3://nasanex/NEX-GDDP

MOD13Q1 (Vegetation Indices 16-Day L3 Global 250m)

Global MODIS vegetation indices are designed to provide consistent spatial and temporal comparisons of vegetation conditions. Blue, red, and near-infrared reflectances, centered at 469-nanometers, 645-nanometers, and 858-nanometers, respectively, are used to determine the MODIS daily vegetation indices.

The MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's Advanced Very High Resolution Radiometer (AVHRR) NDVI products and provides continuity for time series historical applications. MODIS also includes a new Enhanced Vegetation Index (EVI) that minimizes canopy background variations and maintains sensitivity over dense vegetation conditions. The EVI also uses the blue band to remove residual atmosphere contamination caused by smoke and sub-pixel thin cloud clouds. The MODIS NDVI and EVI products are computed from atmospherically corrected bi-directional surface reflectances that have been masked for water, clouds, heavy aerosols, and cloud shadows.

Global MOD13Q1 data are provided every 16 days at 250-meter spatial resolution as a gridded level-3 product in the Sinusoidal projection. Lacking a 250m blue band, the EVI algorithm uses the 500m blue band to correct for residual atmospheric effects, with negligible spatial artifacts.

Vegetation indices are used for global monitoring of vegetation conditions and are used in products displaying land cover and land cover changes. These data may be used as input for modeling global biogeochemical and hydrologic processes and global and regional climate. These data also may be used for characterizing land surface biophysical properties and processes, including primary production and land cover conversion.

Available at s3://nasanex/MODIS

Landsat GLS (Global Land Survey)

In the past, the U.S. Geological Survey (USGS) and NASA collaborated on the creation of four global land data sets from Landsat images: one from the 1970s, and one each from circa 1990, 2000, and 2005. Each of these global data sets was created from the primary Landsat sensor in use at the time: the Multispectral Scanner (MSS) in the 1970s, the Thematic Mapper (TM) in 1990, Enhanced Thematic Mapper Plus (ETM+) in 2000, and a combination of TM and ETM+ in 2005.

Available at s3://nasanex/Landsat

Learn More

Education Grants Program

Educators, researchers and students can apply for free credits to take advantage of the utility computing platform offered by AWS, along with Public Datasets such as the NASA NEX data. If you have a research project which could take advantage of the hosted NASA NEX data set, you can apply for an AWS Grant.

NEXT

Page statistics
596 view(s) and 13 edit(s)
Social share
Share this page?

Tags

This page has no custom tags.
This page has no classifications.

Comments

You must to post a comment.

Attachments