Predictable Wireless Networking and Collaborative 3D Reconstruction for Real-Time Augmented Vision   (NSF US Ignite; PI: Hongwei Zhang; Co-PIs: Jing Hua, Jayanthi Rao, Anthony Holt)

Accounting for over 80% of human-perceived information about the physical world and as a foundational mechanism for autonomous vehicles to "observe'' driving conditions, vision is a critical window into the physical environment for both humans and engineered systems. Natural human and machine vision, however, is subject to inherent physical constraints such as being limited to the line of sight (LOS). In road transportation, such vision constraints lead to stress, inefficiency, and accidents in driving, and making turns with obstructed views is a major cause for about 2.5 million intersection accidents in U.S. every year. To transform human vision and machine vision beyond the line-of-sight constraint, this project proposes to leverage multiple visual sensors to enable humans and engineered systems to see-through obstacles, thus transforming the ways humans and engineered systems interact with environments. To this end, this project develops the wireless networking and 3D vision foundations for real-time wireless-networked augmented vision which holds the potential to enable drivers and vehicles to see through obstacles.

Eliminating the LOS constraint of natural human and machine vision and enabling non-LOS surrounding sensing, the developed augmented vision system will not only transform the experience, safety, and comfort of human driving, it will also serve as an important building block for human-in-the-loop autonomous driving and fully-autonomous driving. The developed technologies are also broadly applicable to domains such as public safety and disaster response, thus having positive societal impact. This project also integrates wireless-networked augmented vision research with the cyber-physical-systems graduate programs at Iowa State University and Wayne State University, and it uses augmented vision research to enrich undergraduate education and research as well as K-12 outreach.

Plenary demo & interview at 2017 US Ignite Application Summit 

Publications (Selected):

Broader Impacts (Selected):
  • PI Zhang has shared project results and insight as a panelist at the ``Innovation in the Smart Rural Ecosystem" Panel of 2019 NIST Smart and Secure Cities and Communities Challenge Expo, and at the ``C-V2X for Future Automated Driving and Cooperative ITS" Panel of the 2019 IEEE International Conference on Communications (ICC).
  • PI Zhang has developed a new graduate course "CPR E 548: Cyber-Physical Systems Networking" at Iowa State University, to address the unique networking needs of cyber-physical systems and their applications in smart agriculture, smart transportation, Industrial 4.0, and smart energy grid. The course has been being offered regularly since fall 2018.
  • PIs Zhang Hua have led the establishment of the cyber-physical systems (CPS) graduate program in the College of Engineering, Wayne State University. Besides Wayne State University, the program is highly supported by industry partners such as Ford, GM, Lear, Magna, ODVA, and Automation of Things.
  • PI Zhang has created and taught the new course ``CSC 5260/ECE 5260: Introduction to Cyber-Physical Systems" in winter 2017 that focuses on introducing the technology foundations of cyber-physical systems to graduate students and senior undergraduate students. This course examines a wide range of topics including modeling, design, analysis, and implementation of cyber-physical systems, dynamic behavior modeling, state machine composition, concurrent computation, sensors and actuators, embedded systems and networks, feedback control systems, analysis and verification techniques, temporal logic, and model checking. The course helps prepare graduate and senior undergraduate students for pursuing advanced topics in areas such as connected and autonomous vehicles, Industry 4.0, Internet of Things (IoT), and smart and connected health.
  • PIs Zhang and Hua have been actively training Ph.D. students involved in the project, through regular research group meetings as well as individual meetings and discussions.