Two Eyes, 3D

Free-Throw Shooting IRL or TV

I ran across an interesting statistic in the book Scorecasting. The authors had dedicated a chapter to whether there is a home/away effect in sports (there is, but not for the reasons you think). They mention that the percentage of free-throwing shooting in basketball does not change between teams playing at home or on the road. After about 20,000 games being analyzed, the free-throw success rate was 75.9% for both home and away teams.

That surprised me. I’m not a huge fan of basketball, but I do recall seeing scenes like this where fans go crazy trying to distract free-throw shooters:

I can understand how these tricks may not work very well, but to not work at all? With a sample size of many thousands of games your statistical power is pretty great. You can usually detect anything you want. Yet the success rate is the exact same (down to  tenths of a percentage). The fans have no impact on the shooter’s ability to deliver the basket. Why not?

It got me thinking about 3D. When we look at shots like those above, we are seeing them in 2D. There is very few pictorial cues to differentiate the fans and their toys from the basket. While, in real life, the basketball player has the benefit of depth perception to separate the basketball net from the fans. Thus, the television/computer screen is amplifying the distraction by flattening everything into one plane.

I’d like to watch a basketball game in 3D to test this theory. Or better yet, get tickets for the Garden. Sadly, tickets for 2 are probably more than a nice 3D TV these days. I may have to stick with the glasses.

Steady as she goes

Spring is the travel season for science education researchers. The American Education Research Association (AERA), National Association of Research in Science Teaching (NARST) and National Science Teacher’s Association (NSTA) are all usually in March or April. Also, the NSF’s Informal Science Education (ISE) division has a semi-annual conference ISE PI’s are expected to attend - and it is also typically in March. So not much actual project development work gets done. I was gone two out of the last four weeks and the other two weeks are making up for lost work in the day job.

Still, I did visit Worcester Polytechnic Institute to give a talk on the project to their Learning Sciences department. And today I gave a virtual talk to a spatial cognition class at Penn State that focused on this project. Part of running these big, grant-funded project is “flying the flag” - giving talks, presentations posters, etc. to let the greater community know what you are doing. My graduate advisor used to say that research is only turned to knowledge when it is shared.

I was talking to one of my Co-PI’s today about the ipad instrument we are using for the pre- and post-tests around the films. We are discussing item formats. We all know that multiple choice items are the worst type of test item ever. They are very, very coarse measures of knowledge - at best. But not everyone is comfortable using an ipad to write out a few sentences or a paragraph. She suggested we have them answer the questions with their voice.

Brilliant! My head was stuck in the paper-and-pencil past and I was not using the full capabilities of the iPad. I think what we’ll do is give the person the option to type an answer using the touch pad keyboard or “click here to speak your answer”. Of course, that means someone will have to transcribe the responses. But one step at a time. :)

And so ends the most boring post in the project. I promise things will get interesting soon as we begin developing real products and testing stuff, i.e. data!

Producing a Show

Last week was an interesting learning experience visiting the Space Visualization Lab at Adler Planetarium. Four of us on the Two Eyes, 3D project met there to discuss the first film and methodology. I have never been involved in development of a planetarium/science visualization show before, so I was very interested in the process.

It began before the meeting when I assembled the stereoscopic design principles we plan to use. I also wrote a first draft of the script for an 8 minute show on Type 1a Supernovae. I had no idea where to start so the script was truly a stab in the dark.

We began by talking about the principles. I was happy to hear from some very distinguished and experienced producers/directors that some of the principles are very much common sense in the industry. That doesn’t mean that everyone follows them! One example given was the recent movie Owls of Ga’Hool, where the director said he sought to break all the 3D rules when making the film. Still, we had many times when fidelity to the principles meant making some changes to the script.

After discussing the principles, we went over the script. We decided to narrow the focus from Type 1a supernovae in general to SN 2011fe in particular. It is the brightest supernovae in over 20 years and, as such, provided a ton of data to astronomers trying to figure out exactly what causes them. There is an interesting narrative story behind SN 2011fe that we can exploit. Our only major concern is that, since it is topical, the science could change quickly over the next few years as the data is analyzed and published. So we’re taking a risk aiming at a moving topic!

I went to the hotel room in the evening and rewrote about 70% of the script on my iPad. The next morning we went over the new script, changed a few things (such as the opening) and story boarded it. Below are some images from our excellent storyboard artist, Julieta Aguilera. The next step is to polish the script this week and send it back. Then the Adler folk begin production. My next step is to direct a recording session  for our scratch track narration this May. We’ll reconvene in mid July and do some pilot testing of the research methodology and focus group testing of the film. The premiere is tentatively set for next November or December.

Regarding the methodology, we also came up with a draft assessment instrument design and blocked how we will recruit and give the assessments.

So there is still a lot of work, but we got a lot done. I was worried about some aspects of the creative process, having not done anything like this before, but it went well and we got more done than expected. Now I have to get back to working on study #1, which I don’t think I’ve described yet. That will be the next post…

Below is a video of Dr. Mark SubbaRao, one of the Co-PIs on the project and the Director/Producer of the films we are making. He’s talking about gravity and galaxies using a simulation he created. Our second film will be on this exact topic.

Towards Stereoscopic Design Principles

I’m heading to the Adler Planetarium this weekend to work on scripts, treatments and storyboards for our two films. We’ll also be checking out the space and brainstorming over some research methodology questions.

In preparation I read 83 evidence-based papers on using stereoscopy to convey information. The goal was to distill them into a set of design principles we should use to design our films. Remember, these are educational films about galaxies and supernovae. Entertainment is important, but only in how it assists learning. So we are less interested in “awe” than in thoughtfulness.

I’ve come up with four principles that we’ll work with. It’s definitely a work-in-progress and I expect them to change by the time the project is completed. More papers will be found, discussions with practitioners and theorists will bring up new ideas, and of course our data will have the final word. But it’s a good start.

I can’t share them publicly here because, frankly, I put a lot of work into this so I feel some ownership of the I.P. I’d hate for someone to take the work and publish it as their own. However, I’m willing to share them via e-mail with people who are working on like minded stuff. Just drop me a note.

I’ll report from Chi-town in a week or so.

Quantitative Assessment of 2D versus 3D Visualisation Modalities

This paper was recently published by Sarah Ting, Tele Tan, Geoff West, Andrew Squelch, and Jonathan Foster, most of whom are with Curtin University in Australia. Their goal was to “…to quantify and compare the human brain’s response to 2D versus 3D images using EEG technology.” EEG stands for Electroencephalography, which is a process of monitoring electrical activity in the brain (using equipment like in the picture below). I don’t have a background in EEG so cannot comment on their procedures. However, their research design and results are interesting. 

They showed a person a series of images that were the same (such as a series of squares) and then included a nonstandard image (such as a circle). This “oddball” is well known in the field to generate a specific electrical response in the brain. For this study, instead of squares they used stereoscopic cubes. And instead of circles they used stereoscopic spheres. Finally, they created 3 versions of each cube and sphere based on how occluded they were. One version showed the entire object. Another version had a cloud-like obstruction blocking 30% of the object. The last version has a cloud-like obstruction blocking 60% of it. 11 subjects were tested with both 2D and stereoscopic versions of the images. Each was given some practice time at the start to become familiar with the stereoscopic system.

They found that the amplitude of the EEG signal they were monitoring was different between the 2D and stereoscopic stimuli. And that difference was related to the level of occlusion. They also found that with at 0% occlusion there is no difference in response rate in terms of time between appearance of the stimulus and the measured signal (in fact, they suggest in the Conclusion that stereoscopic may have been faster). At 30% occlusion there was no difference. However, at 60% occlusion there was a difference between 2D and stereoscopic images, with stereoscopic being more delayed.

This made me happy because it supports the findings in my first paper: stereoscopy  increases cognitive load and the amount of increase is related to the complexity of the stereoscopic image. In fact, that is how this study came across my desk. They cite the paper and say that “[it’s] findings are consistent with this study…” and that “…[our] inference is consistent with [their] findings.” The inference being that the human mind, when seeing things on a flat screen, is conditioned to interpret it in 2D. So even if it is stereoscopic, the brain converts it back to 2D before analyzing the image.

That paper was my first real education research paper, so seeing the results cited and confirmed is exciting to this young researcher. An old hat may be looking at this and laughing, but a first is still a first!

Their research was reported as a pilot study, so I’m very excited to see what this team does next.

                                                  EEG display (Wikimedia Commons)

Poster at American Astronomical Society

The 219th meeting of the American Astronomical Society will be hosted in Austin, TX.  January 8-12, 2012. I’ll be there in force on Tuesday when I 1. give my dissertation talk about citizen science and scientific literacy 2. co-author a talk about a planetarium show we made 3. chair a session on education research in astronomy 4. present a poster on this project and 5. host an evening networking party for those interested in informal science education careers. It will be a long day, but I plan to be fueled by BBQ morning, day and night.

A poster is a large print out describing early research results or announcing something new to the community. The stereotypical poster contains pilot data or early results of analysis of data that will eventually end up as a paper in a peer-reviewed journal. Two Eyes, 3D is too new to have any data yet. So we opted to do a poster that just announces and describes the project with the goal that we’ll find some like minded folk.

You can read our poster by clicking here. But be warned, it is 19MB in size.

Posters tend to be on the order of 4x4 feet, so the PDF files generated need to be at a very high resolution, meaning very large file sizes. It typically takes about a full day to complete a solid poster draft, from start to finish. Then I’ll edit it and pass it around for comments over the course of a few days before printing at a place like FedEx Kinko’s for a whopping price of around US$125.

This poster is a little boring because we have neither data nor analysis to report! But I did throw in some pilot data results so hopefully there will be something new for fellow stereoscopic researchers. I’ll likely be making these types of posters for many conferences over the coming 3 years of this project. I’ll post each one here as they are prepared.

As for the project itself, this month has been spent mainly on logistics. I visited the Boston Museum of Science again and scouted out a place and tentative date to begin data collection (summer, 2012). Our grants officer has been working with them and the Adler Museum to get various contracts and payments in place. And the advisory board sent me some nice feedback on the spatial cognition assessment (see last post). After a 2-week holiday vacation, I plan to focus on assembling the stereoscopic design principles that we will use as our guiding lights for production of the two films. That will be fun and I’ll be sure to share them with you here.

Spatial Ability and Stereoscopy

One of the core questions of our research is whether stereoscopic learning is related to prior spatial ability. For example: Do those with low spatial ability have more trouble learning from 3D visuals? Or does it help them more than those with high spatial ability?

Theory exists to support both hypotheses. Some studies show that stereoscopy increases cognitive load because there is more for your mind to process. Anyone who feels overwhelmed or gets headaches from 3D presentations may feel that way (though headaches could be physiologically related, of course). There are plenty of learning studies that support this, including my first study and this recent study from the University of Texas at Arlington. OTOH, there are studies that show stereoscopic presentations actually lower cognitive load. The fact that pilot and surgical training has used stereoscopy for decades is based on this assumption. You want a pilot to have as realistic training experience as possible, so when they are in a crisis situation their mind can retrieve and apply their training experience as quickly and easily as possible. Stereoscopy may help by minimizing the mental transformations the pilot/surgeon has to perform.

So which is it? I personally think it has to due to cue theory. It’s effectiveness is based on how the stereoscopic images are used in relation to other forms of depth cues. That is why we are assembling a set of stereoscopic design principles to apply to our 3D film productions. But I have no evidence to back this up yet. It’s just a hunch based on theory and prior experience. We are doing these studies to test the hunch.

So we need to measure the spatial ability of people watching the films and slideshows. How? There are lots of spatial cognition instruments out there. There is also great debate as to what factors go into general “spatial ability”. I settled on “spatial visualization” as the factor to measure because:

1. it seems to be the least controversial of the various factors of spatial ability

2. it has a high correlation with overall spatial ability

3. there are learning studies showing a correlation with learning contexts (chemistry and geology) 

4. it seems to be conceptually most similar to where I think the difference between 2D/3D lies (in how the brain processes multiple, simultaneous depth cues).

So we plan to use the Purdue Visualizations of Rotations Test. We will pilot it this spring with children at the Boston Museum of Science and next summer with adults at the Adler Planetarium. If you see someone walking around with an iPad and 3D glasses on their head - say hi! :)

Sample block rotation item.

ASTC 2011

I’m in Baltimore for the 2011 meeting of the Association of Science and Technology Centers. They are basically the main trade group for science museums, planetariums, aquariums, zoos and just about any informal science organization. They have about 1700 attendees at this conference.

Yesterday they had a morning panel on 3D film development held in the IMAX theater at the Baltimore Science Center. It was a pretty good session for someone like me. I’m new to the production side so it was an interesting glimpse into what goes into 3D film. I know the basics, of course, but my experience is on a completely different scale. For one of the films, they had something like 250+ CPUs crunching for over a week to render just a single model of a star formation region. I thought with modern machines those days were over - apparently not. Moore’s law applies to animation as well as the CPUs that render it.

One person in the audience kept raising a question about fidelity to reality. Let’s say you have a 3D image of the Andromeda galaxy. In real life, the human eye cannot see stereo beyond 50 feet or so. To get stereo images of mountains and such, the photographer has to physically move the cameras far apart to create the parallax needed. For galaxies, this is simply impossible. Even if we had Star Trek/Wars type of technology, we wouldn’t be able to do it. The distances are just too great.

So are we lying by showing such an image? The panel basically said “Yes!”. They claimed artistic license is needed to keep it entertaining. I think there was only one astronomer on the panel, the rest were primarily visualizers and directors. But I know some of them personally and I know of the others. They are actually careful to be faithful to the science, where possible. I think they were limited in their responses because the session was running out of time (since we were in a theater, we are unable to “run over” because other films were scheduled). So kudos to them for being honest, but I think the reality is more complicated.

My take is that whether you are being faithful to truth or not is based on the learning goal of the film. Is the goal to show what something would look like to the human eye? (As would be the case in training video, for example.) In that case, the stereoscopic galaxy is indeed a lie. But what if the goal is to educate about the structure and properties of galaxies? In that case, then the stereoscopic galaxy is not a lie because the 3D nature of the galaxy is real. That is, the galaxy would definitely look like that from a specific viewing angle in the Universe. It’s just not a viewing angle we’d ever be able to see ourselves. The spatial properties of the central bulge, the flat disc, the orbiting globular clusters, etc. They all exist. They were not created for the film. So, for me as a learning scientists, if what you see exists in reality then it’s game.

After the discussion we saw a new 3D IMAX film by National Geographic called Flying Monsters, narrated and written by Sir Richard Attenborough. I thought it was pretty good and effective at teaching a light lesson on evolution. Some of the stereo was very nice, but some of it was way too strong and ghosting was bad (I was in the sweet spot of the theater too). I REALLY hate the opening shot, which is a super stereo animation of the National Geographic logo. It almost caused me to throw off my glasses and is a classic example of why you should ease people into the 3D, not throw them into it to get a cheap thrill. It sucks that this will apparently be at the start of every National Geographic 3D film made.

First major study of 3D in the classroom? Not really.

Wired has an article about a large-scale study of the use of 3D in the classroom in Europe. Overall, I agree with pretty much everything the author says. But I want to expand on some of it.

First, this study was funded by Texas Instruments, a major player in the 3D field. They stand to gain lots of profit from increased adoption of 3D technology. Also, the study was not published in a peer reviewed journal. Instead, the results were published in the form of a white paper and YouTube video case studies. The white paper reads as a 6-page marketing brochure and gives almost no information on how the study was conducted. It certainly does not include enough information to judge its quality. That does not by itself mean the results are bad. But it means we can’t truly evaluate them because the details are hidden. In other words, this is not a scientific study. So take it for what it’s worth…

Secondly, the Wired commentator mentions the effect of novelty on the experiment. That is, students may learn more simply because they are paying closer attention due to the 3D effect. I am glad he mentioned this as it is often ignored in stereoscopy studies. I think it has a major, major impact and has to be controlled for. In our study, we intend to control for it by having all students wear 3D glasses in all experiments - regardless of whether they are seeing the visuals in 3D or 2D. So, at least in the beginning (and in my experience likely the entire time) the students will think they are seeing 3D whether they are or not.

Still, this Texas Instruments study is a useful one. Why? Because it is the first large scale experimental study of this type I’ve yet seen in the literature. So it at least offers some insight. Even though it isn’t scientific and thus cannot be trusted on its face, it can be used as a sort of thought experiment to think about things differently. For example, the white paper says “The 3D pupils were more likely to use gestures or body language when describing concepts.” (p. 3) I noticed this as well in my first study and that is why we are doing our assessments for this project using iPads. We are going to use the accelerometers to follow how students hold the device as they answer questions as a measurement of embodied cognition.

There is a funny statement in one of the videos:

“There is never any behavior problem [when using the glasses]…”

Hah. In my experience in an 8th grade classroom, the kids were indeed a little more subdued when wearing the glasses, but they were by no means perfect. There was plenty of joking around, picking on each other, etc. All the typical things that go one in a classroom were still going on. This was completely a marketing maneuver.

The TI web site says this is “pilot” data. So I wonder if it will ever be published. It presents the head researcher as the “Director of the International Research Agency”. I have not heard of that before, and Google-fu turned up zilch. So I question the legitimacy of that organization. I did find the CV of the researcher, and it is impressive. But it’s in the field of art education, not cognitive science. Again, this does not mean the research is bad. It just means that I put the chance at ultimate publication at less than 50%. Although I sincerely hope that I’m wrong. If published, it could be a nice contribution to the field.

Study #1 At a Glance

Our research consists primarily of two main studies.

The first is an experimental study involving a few hundred children at the Living Laboratory in the Museum of Science, Boston. The LL is a neat space. It is setup for cognitive researchers to conduct small-scale experiments with children in the presence of their parents. The researcher is required to be present and must explain their research to the parent while the child is occupied by the experiment. The goal of the LL is to educate the public as to how scientific research is done – and what better way than to show them an authentic experiment underway in front of their own eyes?

I am going to setup an Alioscopy stereoscopic display of around 22”. This is a glasses-free display running on an OS X computer. We will write a slide show program (in java) that will randomly show the child 2D and 3D slides of highly spatial scientific objects (clouds, mountains, crystals, etc.). We will give them an iPad to answer questions about spatial properties of the slides. Questions may be like “which tree is further, the black or brown one?” or “how many clouds do you see in this picture?” The last item will be a drawing task where we ask them to draw something they saw from memory using their fingers. This item will be graded not by accuracy but by the number of spatial elements they include.

Before the slide show begins, they will be given 5-10 spatial cognition questions to help establish a rough measure of prior spatial ability. We will be using items from previously published tests that involve tasks like mentally folding pieces of paper.

While the experiment is underway, I will talk to the parents and will ask them about how much experience their child may have with computers, video games and 3D videos.

We will be looking mainly at differences in accuracy between the 2D and 3D visualizations. We’ll use ANCOVAs to see if any differences are related to prior spatial ability, gender or age. This is a relatively straightforward experimental design (which is the way I like it for early studies!). We will pilot test it this summer and there may be a lot of changes, so this is all in flux. We hope to begin taking data next fall for a 3-month period. If you are in Boston – stop by!

I’ll describe the 2nd study in the follow up post. It’s the sexier of the two since it involves making High-Definition stereoscopic movies. ☺

Full Speed Ahead

It is done. The NSF has formally awarded us the grant to conduct our research. The “Two Eyes, 3D” project involves two studies: one with children at the Boston Museum of Science and another with adults at the Adler Planetarium and Astronomy Museum in Chicago. In the coming weeks I’ll describe the preliminary research design for both studies and begin summarizing some background literature on stereoscopy and spatial cognition.

The goal of this blog is to shed light on the research process so I hope to describe all the steps along the way - good and bad. It will be something of a sounding board of ideas, some of which will surely be poor and (hopefully) tossed aside over time. But some will likely work out and end up telling us something new about how stereoscopy can be used to teach highly spatial scientific concepts (or not).

The grant runs for 3 years, so I hope we are all around on its last day September 30, 2014. This should be a fun ride!

Jumping to Conclusions

There has been a lot of news in the last couple of weeks about a research study that found 3D movies do not enhance enjoyment, cause stress-related physical strain and do not help memory. But let’s take a closer look before jumping to the conclusion that 3D in movies sucks.

First, the study was presented at an industry conference and has not been published in a peer-reviewed journal. That doesn’t mean it is poor, just that it has not been checked out and, more importantly, it means we don’t have the details of the study. All have is what was reported to the media at the conference. I’ll quote this article:

Carrier and colleagues had 400 students watch one of three movies in either 2-D or 3-D: “Alice in Wonderland,” “Clash of the Titans” or “How to Train Your Dragon.”

After viewing the film, participants went home and completed an online survey. They were asked to rate how realistic the movie was to them and to report emotions and sensations they experienced, which they selected from a list of 60 words. The words ranged from mild emotions, such as “enjoyment,” to more intense ones, including “anger” and “rage.” Participants were also quizzed on their knowledge of the film.

The survey showed that neither group remembered the movie better than the other. It also showed that 3-D movie-watchers did not experience a greater sense of immersion in the movie’s world, nor did they pay more attention to the film or report experiencing more intense emotions.

I have a number of beefs. First of all, self reported studies always should be taken with a grain of salt. Sometimes in psychology that is all you have because studying the human brain (emotions, etc.) is quite difficult. But such research design is not very sensitive, mainly because there is no zero point. That is the subjects are not on the same scale. For example, what “anger” means to one person is different than it means to someone else. Thus, the difference between “anger” and “rage” can be different between people. Secondly, the sample sizes are small. You have 400 people divided into two groups, with each group divided into 3, thus you have about 65-70 people per film per condition. Unless the results were very strong, I doubt it would be statistically significant. Again, we’ll be able to check for all this if the paper is published (a high percentage of conference presentations never make it to publication - I’d put it at around 50%). Third, how can one judge whether a cartoon (such as How to Train Your Dragon) is more realistic in 2D or 3D?

I could go on and on but will stop because it is somewhat unfair to criticize research at a presentation I did not attend. But it was publicly released and the authors have made free with the media, so it is fair to ask questions.

The authors say: “”It didn’t seem to enhance your memory at all,” Carrier said. “That’s an unfortunate implication.”

Again, how did they determine that? Without an experimental study I just don’t see how they can say that based on the study design described in the press reports. In science the importance of a claim needs to be correlated with the strength of your evidence (as Carl Sagan said: “extraordinary claims require extraordinary evidence”).

I’m not a 3D champion. My first published study on this subject actually agrees with their core result! In my experience, 3D increased cognitive load and did not have any effect on accuracy of spatial cognition tasks I gave the subjects. And I buy the argument that 3D may not increase enjoyment of a film. Anecdotally I think audiences are starting to show that. And I think this study adds a new data point to the scholarly pool. But I think it’s a minor data point and it does not mean what the press says it means. The evidence is not enough to say that 3D increases discomfort and does not affect memory. Not yet.

Rebirth

So a lot has happened in the 2.5 years since my last post (at Blogger). The silence was caused mainly by my dissertation topic changing to something that has nothing to do with stereoscopy. But I’m now done with that, have graduated, had a child and received a promotion at work. But stereoscopy never left my mind or heart. And I also recently got an NSF award to spend the next 3 years doing research on stereoscopy in museum settings. More on that later…

2.5 years may not seem like a long time. But in the world of consumer electronics it is an eternity. Read some of these old blog posts. They mention “an upcoming movie called Avatar”. Who knew that it would go on to become the highest grossing film of all time? Now 3D is everywhere. But mostly it’s not very good.

That first stereoscopy research paper was published in the meantime. And I did some pilot work on my original dissertation topic. I’ll share that research here as well. Stay tuned!