Jessica Colnago

I was a teacher for the Summer Immersion Program in New York City, sponsored by IBM. In this program I taught basic CS concepts and some programming languages to 20 high school girls in order to introduce them to the world of CS and motivate them to pursue CS as their major in college.

At HCT I worked with wearable technology and participated on a project that focused on exploring the theory of proxemics applied to wearables. I performed research, built and tested prototypes using a web-based platform and RaspberryPi, and prepared a user study protocol for later use by the group.

At LIA I worked on the Breakout project, a public space game with full-body interaction, and the Wishboard project, an interactive public space art installation, where I analyzed data and worked on papers with graduate students. I also worked on their wearable project, a series of design prototypes for mediating the interaction between user and a distant community, doing research, building prototypes using Arduino-based and RaspberryPi boards, and big screen web-based interfaces, as well as helped advise a bachelor thesis.

During my time at Embrapa I was assigned to work on the FACE Brasil system which was associated with a project to study the effects of climate change on agriculture. My work was related to the data collection side of the project and, as such, I developed the database for the FACE Brasil system. This involved the database design and implementation, development of user-friendly query tools and database management tools, as well as script development for data migration.

At the Intelligent User Interfaces group I worked on several tasks associated with an existing web-technology based mobile multimodal platform and the prototype Android app Voice2Social. I improved their dialog manager (SCXML) responsible for receiving and fusing multimodal inputs; implemented the "Lift-to-Talk" functionality to trigger the speech recognition for voice commands; and, implemented additional features and optimized the "Sound Radar" functionality, which allowed users to query restaurants nearby based on sound related filters (e.g. music style). As a side project I implemented a speech synthesis component for the SATIN visual programming editor.

For my internship at the Q&U lab I was assigned to the "Smart Senior" project. I worked on the INSPIRE system, a Java based multimodal system for Smart Houses, for which I worked on the multimodality input fusion module, as well as implemented gesture and speech recognition, using Microsoft's Kinect and Google's Voice to Text API respectively. To test these functionalities I ran a small scale user usability study.

I assisted Dr.-Ing. Sebastian Feuerstack, at the time a PhD student, on his project involving the use of state machines to ease the development of multimodal user interfaces. I developed a parser that read SCXML state machines and generated equivalent Ruby state machine, building upon the existing state_machine module for that programming language.