Wide Area Persistent Surveillance (WAPS) systems that image city-size fields of view and instantly search through terabytes of data are moving from combat deployments to commercial services. Vigilant Stare team members Sierra Nevada Corp. (SNC) headquartered in Sparks, Nev., and ITT Exelis in McLean, Va., are negotiating their first fee-for-service agreements, offering customers the same WAPS sensor flying on the Air Force unmanned Reaper repackaged in a civilian Twin Otter. With five electro-optical and four infrared cameras, the day/night sensor captures a field of view 16 square kilometers. Consolidated Resource Imaging (CRI) in Grand Rapids, Mich., quietly leveraged its extensive WAPS operating experience in Iraq and Afghanistan to pursue commercial customers. The company’s proprietary 116-megapixel LodeStar electro-optical sensor flies on a Gipps GA-8 or other aircraft. Both WAPS systems make processed intelligence products available to multiple users through web-based display devices including tablet computers and smartphones. With near-realtime and forensic (after-the-fact) processing, their geo-rectified pictures can support border security, drug interdiction, traffic analyses and other applications. According to CRI owner and President Nathan Crawford, “A wide-area persistent data set is worth a thousand uses, and we haven’t yet discovered them all.”
CRI demonstrated the domestic potential of WAPS to the U.S. Coast Guard in 2010, monitoring the progress of the Deepwater Horizon oil spill cleanup. “We were on a rolling geo-registration line around the Gulf of Mexico monitoring the oil spill, booms and ships involved with the recovery operation,” said senior program manager Gregg Wildes. “The Coast Guard’s feedback was our aircraft with a single wide-area sensor could replace more than 10 helicopters with smaller cameras.” CRI can provide intelligence products tailored to disaster response, market research, event overwatch, and migration or pattern-of-life studies. Most WAPS business remains tied to the Department of Defense (DoD), but according to Crawford, “Within the next few years, it won’t be. It’s going to go full commercial.”
Wide Area Aerial Surveillance (WAAS) systems went to war on manned and unmanned aircraft and tethered aerostats to counter Improvised Explosive Devices (IED), not by detecting the hidden bombs but by recording activities associated with them. Logos Technologies in Fairfax, Va., made the aerostat-borne Kestrel system now watching over Forward Operating Bases in Afghanistan. The company demonstrated both the Kestrel and Lightweight Expeditionary Airborne Persistent Surveillance (LEAPS) sensors for the Department of Homeland Security.
John Marion, director of the Logos Persistent Surveillance Division, worked on the earliest WAAS developments and explains, “Imagine that you’ve got a Google Earth view of a city-size area and you’re running a couple of times a second and storing all that. When you get an intelligence cue from something an ELINT [Electronic Intelligence] hit, or EOD [Explosive Ordnance Disposal unit] finds an IED you can go back in time and figure out where things went. It’s the way in which you connect the other isolated intelligence pieces.”
Pattern-of-life analyses enabled allied forces to track bomb makers after kinetic events and correlate telltale signs before attacks. WAAS becomes WAPS with the addition of a long-endurance platform. The MQ-9 Reaper Unmanned Air Vehicle with better than 27 hours’ endurance went to Afghanistan carrying the Gorgon Stare Increment One sensor in 2011. Last year, Vigilant Stare system integrator SNC and sensor supplier ITT Exelis put an identical sensor on a DHC-6 Twin Otter with four hours’ mission endurance. Dave Bullock, SNC vice president of persistent surveillance systems, acknowledges, “Other contracts in the future may require a longer-duration airplane. Each of those configurations will be operational once we have a customer in hand.”
Different WAPS systems all share wide field-of-view sensors and powerful processing technology to search collected imagery. According to Marion, “You’re not collecting it because anyone’s going to look at all of it. You’re collecting it because you don’t know when and where the bad thing is going to happen.”
Technology to geo-rectify WAAS imagery emerged soon after the first datasets were available. “The minute you pinned the imagery down on a flat earth model, suddenly you could see things moving more clearly,” said Crawford. Datalinked to the ground, processed WAAS imagery can follow an individual vehicle through busy traffic. Though it cannot reveal license plate numbers or other fine details, the big picture provides sufficient resolution for “blob-tracking.” Crawford summarizes, “It doesn’t show evidence directly but helps point to where evidence is.”
At SNC, Bullock explains, “The game-changing aspect of true WAPS systems… is you have to be a synoptic Wide Area Persistent Surveillance system in order to do forensics in real-time. You see something happen, and you can rewind in real-time to track back and find where things came from. This provides true context that’s otherwise missing.”
He adds, “If you’re looking in one place with small field-of-view systems, you’re going to miss things. With this wide-area system, you’re not going to miss things.”
In Context
Today’s WAPS technology is rooted in Department of Energy efforts a decade ago at the Lawrence Livermore National Laboratory in Livermore, Calif. Scientists working on the Sonoma Persistent Surveillance Program used commercial imaging technology to build wide-area sensors in successive steps from 2 to 66 megapixels from 2002 through 2004 when the emphasis transitioned to the DoD and launched parallel WAAS developments.
As manager of the WESCAM entertainment division in California, Crawford had ample experience integrating cameras into aircraft. His spin-off company CRI integrated a 66-megapixel sensor and processing electronics into a pod on an Army OV-1D Mohawk reconnaissance aircraft. The Mohawk Stare system included an operator cockpit display but no downlink. It was tested in the summer of 2005, but with the OV-1 being retired, the Army had CRI integrate the Constant Hawk Quick Reaction Capability (QRC) on the bigger C-23 Sherpa [Shorts 360] cargo aircraft. “We developed the on-board storage capability for a large image,” recalls Crawford. “No one had taken a spinning media hard drive to altitude the Shorts 360 was an unpressurized aircraft.”
The Army Product Manager for Airborne Reconnaissance and Exploitation Systems deployed the Constant Hawk QRC to Iraq in 2006 in support of Task Force ODIN (Observe, Detect, Identify, Neutralize) to counter IEDs. U.S. forces in Operations Enduring Freedom and Iraqi Freedom found high-resolution, narrow-field-of-view targeting sensors could not provide the big picture they needed to fight mountain and urban insurgencies. Constant Hawk used six 11-megapixel Kodak focal plane arrays for wide-area coverage.
Contractor Jorge Scientific operated Constant Hawk Iraq, a deployment including four C-23s and a ground analytical team for image Production, Exploitation and Dissemination (PED). Constant Hawk Afghanistan in 2009 initially deployed three wide-area sensors on pressurized, higher-flying C-12 Huron/King Airs. A fourth aircraft added a datalink, satellite communications capability, and a high-resolution WESCAM MX-15 gimbal with spotting camera. CRI currently has team members deployed around the globe in support of Constant Hawk and other sensor programs.
Separate from the Army WAAS effort, the Marine Corps wanted enhanced realtime situational awareness. The Air Force Research Laboratory in Dayton, Ohio, and Los Alamos National Laboratory collaborated on the Navy Angel Fire and Air Force Blue Devil WAAS systems first deployed in Iraq. An Angel Fire operational assessment in California in late 2006 showed dismounted fighters could be tracked by a wide-field persistent sensor with 0.5 m resolution and update rates of 1 to 2 frames per second; full-motion video refreshes at 30 frames per second.
Angel Fire went to Iraq on King Air turboprops. Moving masses of information to the ground remained a technical challenge, but ground station analysts could steer and zoom the airborne sensor and see the wide-area picture with 10 seconds latency. Angel Fire analysts could also call up smaller data packets that saved downlink bandwidth, and data stored on the ground could be “rewound” to reconstruct events.
While the Air Force Research Laboratory pursued its WAAS technologies, the Big Safari Program Office of Air Force Life Cycle Management Center in 2006 began parallel development of the Gorgon Stare sensor for the MQ-9 Reaper Unmanned Aircraft System (UAS). SNC was the system integrator. ITT Exelis Geospatial Systems supplied the EO/IR sensor and processing subsystem firmware. Gorgon Stare Increment One went to Afghanistan in April 2011. The system uses five electro-optical and four infrared cameras to provide imagery from 12 different angles and gives ground analysts 50 to 60 high-resolution chip-outs at a time. Gorgon Stare Increment Two with higher resolution and a bigger field of view is in development.
Space, Weight, Power
SNC and ITT Exelis drew on their Gorgon Stare success to build the Vigilant Stare business now in the product definition and maturation phase.
“Essentially, we are leveraging as much of the Gorgon Stare architecture and system as possible,” says Bullock, “but we are tailoring it to the new venues it could have application to a manned aircraft and commercial dissemination.” While unmanned platforms cannot use U.S. National Airspace without prearranged Certificates of Authorization, the Vigilant Stare Twin Otter can react quickly. “We thought a manned aircraft made a lot of sense, not to mention there are not a lot of spare Reapers lying around,” observes Bullock. “Typically speaking, the manned aircraft allows you to operate in civil airspace without prior notice. As long as you file flight plans and obey the rules of the road, you can fly anywhere.”
The Twin Otter was the platform for Gorgon Stare Increment One testing and provided a familiar, affordable airframe with room for Vigilant Stare electronics and two operator stations. One console is occupied by an SNC system manager, the other by a customer mission commander. The Twin Otter is big enough for changing equipment suites and can store imagery on-board to work without a datalink. According to Bullock, “Synoptic collection is one of the things that sets Gorgon Stare and Vigilant Stare apart from any other system. Every pixel on the ground is being collected all the time.”
The Twin Otter typically flies at 18,000 ft but can drop to 12,000 ft to improve resolution with a reduced Field Of View. “If you want to cover a wider area in a maritime case where you’re looking for much larger targets, like boats, we can fly a lot higher and cover a lot more area and still get mission-relevant resolution,” explains Bullock.
Vigilant Stare can use the government-standard Common Datalink (CDL) to send wide-backhaul data to users on the ground, or commercial 3G and 4G networks to send users high-resolution sub-views or chip-outs via the internet. “We did a point-to-point datalink using 3G to do a proof of concept. We have an air node and a ground node point-to-point. We use commercial providers with whom we have a contract. We have to buy bandwidth to get to the ground.” Chip-out resolution is optimized for the military Remotely Operated Video Enhanced Receiver (ROVER). “While you can put the full image through that commercial channel, it’s limited by the display device,” says Bullock. “We are not able to push down the same full-field of view resolution that we get on Gorgon Stare to a commercial tablet just because of physics. There is no free lunch.”
Space, weight and power (SWaP) are key challenges for airborne WAPS systems. Over the last two years, CRI has self-funded development of its compact LodeStar WAPS sensor to offer fee-for-service operations on the company’s GA-8 or leased platforms. “We continue to utilize our sensor integration and test expertise to support national labs and have now developed our own WAPS system that is available commercially in North America,” says Wildes. “While many WAPS systems require larger aircraft, LodeStar is comparable in size, weight and power to smaller MX-15 or Star Safire [electro-optical] sensors. You can put it on smaller aircraft, providing a quickly deployable and very cost-effective system.”
Crawford observes, “The real challenge was creating an entire system that could run for seven or more hours straight collecting huge datasets without breaking the bank.” The color electro-optical payload fills a 15-inch gimbal and includes a GPS-INS geo-steering navigation system. CRI developed its own image capture hardware and software with real-time orthorectification, stitching, camera correction and chip-out generating functions. The LodeStar system includes an integrated data link and browser-based viewer with analytical tools.
“The way we’re doing our image processing facilitates our path to the Holy Grail of all this which is auto-tracking.” CRI is working on high-performance, low space/weight/power systems to perform real-time image processing with high-definition video and imagery sensors and has been selected to compete for the mini-WAAS technology insertion in the Army’s next-generation Enhanced Medium Altitude Reconnaissance and Surveillance System.
Logos Technologies has developed and qualified a miniaturized, advanced version of the Constant Hawk WAAS system to fit the RQ-7 Shadow UAS. Marion explains, “LEAPS came out of that from the standpoint of recognizing the utility of wide area capabilities and the need to evolve them for tactical UAVs and shrink them for tactical platforms.” The 54-pound system includes a 10 in. diameter gimbal and a paint-tray sized processor. “The sensor itself is an important improvement, but the magic is in that small processor,” says Marion.
The 69-megapixel LEAPS was funded by the Office of Naval Research (ONR) and collects imagery from a city-sized field of view with 10 individual viewing windows. It uses a CMOS sensor more sensitive than Constant Hawk photo sensors. The system sends imagery down through a Tactical Common Data Link (TCDL) and records up to six hours of data on-board the unmanned aircraft. LEAPS was demonstrated for the Army on a manned RC-12 (military King Air 350) and for the DHS on a Cessna Caravan. “The thing that’s cool is when you get the sensors small enough, they can kind of go on anything,” says Marion. “We think that’s going to be a crucial aspect there because most law enforcement agencies already own aircraft and are used to putting video balls on them. We have to match that.”