We have all had the experience of when we are looking for something, we sometimes do not find it. But as soon as we stop looking, we do find it – frequently when it is no longer needed. As a professional scientist I know that this observation has little scientific basis, but I feel I have fallen victim to it more than once during my professional career. While wrapping up a final report recently for a grant on a remote sensing project conducted by my group in cooperation with the crew of the International Space Station (ISS), I discovered quite by accident I had fallen victim to this phenomenon once again involving clouds over California’s tallest peak, Mt. Whitney.
RAMOS – 1995
From the mid-1990s until its cancellation in 2004, I was a member of the American science team for a joint American-Russian program called RAMOS (see “RAMOS: The Russian-American Observation Satellites”). The primary technical objective of this program was to conduct joint research and development with our Russian counterparts on new approaches to improve space-based early warning capabilities. Secondary technical objectives included performing observations of a more environmental nature in an attempt to broaden the appeal of the program beyond defense interests. The plan was to launch a pair of Russian-built satellites into low Earth orbit to observe clouds and other atmospheric phenomena in 3D using a suite of American and Russian instruments operating at wavelengths ranging from the infrared (IR) to the ultraviolet in order to create an improved data base for a variety of defense and science-related applications.
Diagram of the RAMOS satellite based on the Russian Yacht universal space platform. Click on image to enlarge. (Khrunichev)
As the design and architecture of the RAMOS constellation and its ground segment were being developed, it was decided early in the program that a series of field science campaigns would be performed to gather data on various types of cloudy scenes. The goal of these near term experiments was to develop the mechanisms needed to plan joint experiments, coordinate operations as well as share data and analysis results between the American and Russian teams. In the process, these near term experiments would acquire data needed to refine the program’s science objectives and validate performance requirements for the instruments to be carried by the RAMOS satellites. I was one of the key members of the American science team responsible for the processing and analysis of the data collected from these joint experiments.
Our first near term experiment campaign took place in 1995. This campaign involved the Russian Resurs-O1 satellite and the USAF/NASA ARES (Airborne Remote Earth Sensing) aircraft making near simultaneous observations of selected targets. The Resurs-O1 remote sensing satellite was equipped with a range of instruments including the MSU-SK conical scanner operating at long-wave IR (LWIR) wavelengths of 10.0 to 12.5 μm and the MSU-E push-broom imager working at visible wavelengths. The WB-57F ARES was a modified RB-57F strategic reconnaissance aircraft which, for our investigation, carried an IR imaging spectrometer capable of gathering data simultaneously in 75 spectral channels from 1.9 to 6.1 μm. It was the hope of the project management teams that near simultaneous measurements from these two platforms could be used to create a 3D reconstruction of any clouds in the target area.
This is a comparison of the Resurs-O1’s MSU-SK IR image and the MSU-E visible band image taken in support of the RAMOS conjunctive experiment on October 5, 1995. The MSU-SK image (whose contrast has been reversed so that cold clouds appear white) covers an area of 600 by 640 kilometers while the MSU-E image (whose position in the IR image is indicated) covers 55 by 104 kilometers. Click on image to enlarge. (A.J. LePage/Visidyne)
A comparison of a panchromatic IR image swath taken by the ARES aircraft (top) and a false color image from the MSU-E push-broom scanner on Resurs-O1 (bottom) acquired on October 5, 1995 over California’s Sierra Nevada mountains. Mt. Whitney is near the center of the ARES image swath. Click on image to enlarge. (A.J. LePage/Visidyne)
The second joint observation opportunity during our initial campaign took place on October 5, 1995 in the vicinity of Mt. Whitney in California’s Sierra Nevada range. The Russian Resurs-O1 acquired its data around Mt. Whitney at about 17:59:20 GMT (10:59:20 AM PDT) as the ARES aircraft was making a series of three passes over the region with Mt. Whitney being observed during a down-looking overpass at about 18:16:45 GMT (11:16:45 AM PDT). The Resurs-O1 LWIR image from the MSU-SK conical scanner covered an area of 600 by 640 kilometers while the visible band image from the MSU-E pushbroom imager covered 51 by 104 kilometers. As can be seen in the images above, there were no clouds visible in the area of Mt. Whitney.
This is a closeup comparison of the ARES image swath created using data from the ARES IR imaging spectrometer (top) and the Resurs-O1 MSU-E pushbroom imager (bottom) taken on October 5, 1995. The image swaths are about 7.5 kilometers long and are roughly centered on Mt. Whitney. Click on image to enlarge. (A.J. LePage/Visidyne)
Undeterred by the lack of clouds, I still hoped to do a 3D reconstruction of the terrain around Mt. Whitney. Unfortunately, the compromises which had been made by the senior members of the American and Russian teams for this initial joint observation campaign meant that this would not be possible with the data available from Resurs-O1 and ARES. While the MSU-E images from Resurs-O1 covered the area with a pixel footprint of about 51 meters, the IR imaging spectrometer data from ARES had a pixel footprint of about 13 meters but were only about a half of a kilometer wide (the equivalent of only 11 MSU-E pixels) as can be seen in the comparison image above. There simply were not enough pixels available to do anything more than a simple 1D altitude profile along the image swaths. The differences in the wavelengths used also caused issues: the areas covered by snow appeared bright at visible wavelengths observed by Resurs-O1 but dark in the IR data from ARES creating a selective contrast reversal in the images which would confuse automated feature matching algorithms then under development by our team. Combined with the lack of sufficiently accurate pointing knowledge for either data set, a 3D scene reconstruction was simply not feasible (which really was not a major issue at this early stage of the program since the important objectives centered on developing the means to work jointly with the Russians).
This is a comparison of the views centered on Mt. Whitney as it appeared in the visible (top) and the IR (bottom). The visible image mosaic was created from ARES video images while the IR was from data acquired by the ARES imaging IR spectrometer. Each image covers an area of about 4 by 0.6 kilometers. The contrast reversal of the snow covered areas (which appear bright in the visible and dark in the IR) is readily visible here. Click on image to enlarge. (A.J. LePage/Visidyne)
Intent on doing some sort of 3D reconstruction, I went to an entirely different source of data. In addition to the IR imaging spectrometer, the ARES aircraft also carried a video camera with a 5.1° by 6.8° field of view to provide context images of the scene being observed at visible wavelengths. By frame-grabbing images from the video, I was able to stitch the images together into a mosaic as shown above. After a bit of work, I came up with a means of combining different parts of the frame-grabbed images to create a “left eye” and “right eye” stereo pair suitable for analysis as shown in the 3D anaglyphic image below. While the resulting stereo angle of about 2½° was rather small, it was enough to reveal parallax effects in the scene and allowed our early stereo analysis software to produce a convincing 3D reconstruction of Mt. Whitney with the relative altitude accuracy good to several percent. While this exercise was a positive step forward, a stereo reconstruction of clouds would have been preferred.
This is an anaglyphic 3D image (left eye red, right eye blue) of the area around Mt. Whitney created using video images taken as the ARES aircraft flew over Mt Whitney on October 5, 1995. Click on image to enlarge. (A.J. LePage/Visidyne)
A 3D reconstruction of Mt. Whitney based on a stereo analysis of ARES video images acquired on October 5, 1995. Click on image to enlarge. (A.J. LePage/Visidyne)
CyMISS – 2018
Fast forward a couple of decades and I was once again working on a 3D remote sensing project called CyMISS (Tropical Cyclone Intensity Measurements from the ISS). Started in 2014, CyMISS was a follow on to an investigation that was originally part of the RAMOS program’s Fast Changing Events experiment. Using a model of how intense hurricanes and other tropical cyclones are powered developed by Professor Kerry Emanuel of the Massachusetts Institute of Technology (MIT) and extended with inputs from a fellow member of the RAMOS science team, Professor Paul C. Joss of MIT and Visidyne, it would be possible to determine the strength of these storms using the cloud top altitudes (derived by stereoscopy) as well as the cloud top and sea surface temperatures more accurately than existing remote sensing methods in use for the past several decades (see “A New Satellite-Based Method to Determine Hurricane Strength”). For the initial phase of the project, the crew of the ISS has been obtaining dozens of image sequences using a Nikon camera and employing a photography protocol specifically designed to create pseudo-stereo views of the cloudy scene (i.e. 3D views created by a single, moving platform).
This is an anaglyphic 3D mosaic (left eye red, right eye blue) of Hurricane Matthew as seen from the ISS at about 20:06 GMT on October 7, 2016. It was created for the CyMISS project by combining various parts of the 240 red-channel images which have been processed to approximate a common overhead view covering an area of 1,600 by 1,200 kilometers. Click on the image to view the full-size version at a scale of 500 meters/pixel. (A.J. LePage/Visidyne/JSC-NASA)
During this past year, we hoped to extend our 3D imaging technique to the study of other phenomena of interest including wildfires. With the intense wildfire activity it has experienced this year, California was one of our targets for study. Between 18:19:01 and 18:22:00 GMT (11:19:01 and 11:22:00 AM PDT) on August 13, 2018, the crew of the ISS secured a sequence of 240 images which included much of the state of California. As can be seen in the color mosaic below created using these images, much of northern California and Oregon were obscured by smoke. The most prominent smoke plume visible in this image appeared to be associated with the Carr Fire in the Shasta-Trinity National Forest on the west side of California’s coastal mountain range. A long narrow plume of smoke can be seen stretching from the fire and northeast across the Sacramento Valley until it reached the Sierra Nevada Mountains.
This is a color mosaic created from the images taken from the ISS in support of the CyMISS program on August 13, 2018 with state boundaries and coastlines added. Note the smoke from the wildfires in northern California and Oregon. Click on image to enlarge. (A.J. LePage/Visidyne/NASA-JSC)
While I was wrapping up the initial analysis of these data during the preparation of our final report for our previous grant, I performed an inspection of the images to see what other interesting phenomena may have been recorded. While taking a close look at the Sierra Nevada range it dawned on me that the area around Mt. Whitney, shown in the image below, was covered by cumulus clouds – the same area that had been recorded as part of a RAMOS conjunctive experiment 23 years to the day before the October 5 due date of our report.
This is a color image (ISS055-E-136334) acquired from the ISS at 18:21:33 GMT (11:21:33 AM PDT) on August 13, 2018 showing the southern part of California’s Sierra Nevada range as well as the adjacent Owens Valley and Panament Range. The image, which is roughly centered on Mt. Whitney, has been processed to approximate an overhead view which covers an area of 150 by 100 kilometers. Click on image to view a full resolution view with a pixel scale of 50 meters/pixel. (A.J. LePage/Visidyne/NASA-JSC)
I could not resist the temptation of performing a quick stereo analysis of these images during my free time this past month. As can be seen in the anaglyphic 3D image below, the puffy cumulus clouds stood out above the mountains below with couple tall cumulonimbus formations visible including over Mt. Whitney near the center of the image. While this made for a stunning 3D view, it came 23 years too late for the RAMOS program. Now I am wondering how many more times I will be observing clouds in places where I have hoped to see them (and did not!) a decade or two ago.
This is an anaglyphic 3D image (left eye red, right eye blue) matching the color image above. Created using the red color plane of the images with a stereo angle of about 14 degrees, it clearly shows the cumulus cloud formations over the mountains visible beneath. Click image to enlarge. (A.J. LePage/Visidyne/NASA-JSC
The CyMISS team at Visidyne would like to thank the crew of the ISS as well as the staff at NASA’s Marshall Space Flight Center and Johnson Space Center for their ongoing efforts. The original CyMISS images are courtesy of the Earth Science and Remote Sensing Unit at NASA Johnson Space Center. The work presented here is supported in part under CASIS Grant UA-2019-013.
Follow Drew Ex Machina on Facebook.
Related Reading
“RAMOS: Russian-American Observation Satellites”, Drew Ex Machina, June 21, 2014 [Post]
See earlier articles on the CyMISS program here.