29 February 2016

Progress update #2


Got some time to put calibration working in a simple way (I think) and with good results.

To perform calibration we capture images from one or more chessboards.
This can be done using live feed from cameras or from a video file or image sequence.

Results of the calibration progress are presented immediately to the user.

We can evaluate the captures and decide if we want to keep or remove them from the calibration list.
We can have some chessboards incorrectly identified and they must be suppressed from the calibration, but this has to be done manually.

Sources are calibrated separately and the stereo is calibrated in the end.

One of the new features, is the possibility to use more than one chessboard in each image.
The limitation here, is to use same size chessboards with the same square size.

With a good pc everything works in a fast and smooth way. I'm concerned with slower ones, it will work, but takes more time.

Just to post some images on the progress.

Bellow one image of the mono calibration.
We can now calibrate devices individually and in stereo.

The image shows in the left the camera capture with an overlay of the captured chessboards.
On the right the captured chessboards list (we can remove the bad ones if we wish).
On the middle the undistorted result.

I also added a way to capture chessboards from the screen.
Connect the pc to a TV or projector to get bigger calibration patterns and calibrate the cameras without the need to print the chessboards.
This has the advantage of using an already planar surface.

I also got two 360º lenses to test omnidirectional reconstruction.
The calibration seems to work well already.

Some news soon...

Just to add a spoiler...


  1. Niiiice updates are speeding up :P! Wow omnidirectional, that could be hot!.
    Duuuude. GPS corrections might be on the way? thats coool!, there are logger apps that do also record imu data from teh phone, for the future may beee.

    For those with big pockets the BM Micro camera witch has genlock, global shutter and interchangeable lenses might provide amazing ressults.

    Whatever, I'm really curious about how the omnidirectional reconstruction behaves.

    Using a big TV is quite a convenient way to display the checkboards!.

    Can´t wait for updates!

  2. I now understand why you use webcams, because these probably are ccd sensor having global shutter right?

    1. No, I'm using the Logitech C920 and the Hercules HD for testing.
      Both are rolling shutter, and the Hercules is awful.
      But I want to capture in daylight and with those conditions we can use small exposure (I force it manually) making use of the fastest possible shutter speed.
      With the Logitech I get good results, the Hercules, I don't know, I use it just for testing code and for portability (small rig) in stand still.

    2. Interesting. I've done a bit of research and 1080 with CCD sensors did not arrive to the market in mass as CMOS took the lead. There are a couple of cams offering 720p with ccd sensors even with wifi but the quality really seems to suck, the sample vids on youtube are just terrible. ZS10 and surprisingly recent(2014) Canon S200. Having global shutter and being able to launch recording from a phone certainly was promissing but quality is a joke. When I was givin up I found this thing: Pannasonic TS4. CCD sensor and, tadaaa 1080p recording at 30fps. Misleadingly you might find 1080 60p but its i for 60. Higher models do have 60p but with MOS sensor witch I doubt is global shutter.
      But in comparison video the TS4 certainly does not offer super sharp images. Sure youtube compression does not help.

      Anyway if global shutter is the way to go, the TS4 looks like "the camera" for regular budgets, it does even stand water so if starts to rain no worry with your rooftop cams.
      Here a vid with an HX9v I made years ago. I wonder if frame sync is so much important. I could get another HX9v for testing as now probably they're pretty cheap.

    3. forgot teh vid.

    4. I've searched a bit further and found no other 1080 CCD compact. Found some JVC 3CCD cams that are not to expensive recording at 1080. That said I recalled the gopro 3D hero system. I wonder if it does appart from making the second camera slave, it sincs the sensors preciselly. I fi toes it could be very interesting as it could record 2.7K with superwide mode. Or even 4K at 15FPS.

    5. For omnidirectional recording I think this is worth checking!
      pixpro 360 4k

    6. Cool! $271.41 at Amazon.
      I got two BubblePix lenses for around 70€. I'm trying to put them on two Logitech C260... And it will never be 4k.

    7. Thank's for the info on the cameras. I will take a look.

    8. beware with the pixpro as there is the 360 and the 360 4k (newer) model. I'm a bit concerned about this camera as some youtube footages seems super soft but probably is youtube compresion but found no original downloadable sample vids.

      I found another 1080 CCD. Lumix DMC-FZ48EG-K. Its a bit bulkier as it comes with a non interesting superzoom, at least it has a wider 24mm lens.

      Still I think is better the sport cam CCD, less size. I'll try to get on for cheap.

    9. I'll go with Aukey Optic Pro Lens for my lg g4. Sure its worth as it has 4k recording. There are some cheaper Aukey, not sure witch one should take. I'll post

    10. More on Aukey lenses.
      There are abasically three options for wide angle/fisheye from this manufacturer.
      A 3 in 1 kit, super cheap but the fisheye losses sharpness on the edges

      An ultra wide lens, named pro but with high sharpenss loss overall,

      A 110 fov fisheye where manufacturer claims to be sharpe from edge to edge with low distortion etc..

      I'll buy the last one as its a cool gadget too. I'll post an in car video.

    11. Aukey lens was terrible, super soft, returned to Amazon. Got a lumix hi speed CCD camera and looking for extra one with exact specs now. Also aquired a cual camera mount, it has about 30cm separation, not super big but it will be convenient to move from car to home so it can be calibrated easily in television. In a couple of weeks I spect to have the rig built to go out and capture a video.

      My plan is to later on sync the images by matching handclap sound graphs. As the camera outputs HDMI and AV other option would be to captura video directly from a pc. HDMI capturing video cards are pretty expensive and require pci. The micro usb cable outpust to av video so I believe will support usb video interface, may be this could be easyer, but dont know if it will output 1080 this way. Anyway, not sure if the maximun possible shift of 1 frame (global shutter CCD) is worth that much effort.

    12. Finally, I got dual Hi speed CCD cams. I do not have the rig to mount them yet and will take a bit long, I'll try to figure out something while it arrives. Please share the calibrationi patterns of the latest beta build

    13. OK!. I finished my rig and did a test data capture.

      The problem now is that the old PVS that does not allow me to calibrate the video because it espects two cameras to be conected to the pc as it is more oriented to webcam input rather than self storing video cameras. I'll contact you directly to get a beta.

      Also, as the cameras are 1080 the Xvid and Huffyuv codecs are vry limiting as they top at resolutions much lower as it requires SBS video, may be you could allow for left and right video input in separated files.

    14. Hello. Could you release a Beta version of the actual in progress build?. Or simply update the public version to allow separate left-right video file input. I've been able to reconstruct in other packages but sooner or later the track of the road is lost. Also they do not extract as much road points as your app, even if the video feed is at higher resolution.

  3. Few questions:
    What is the depth / surface height accuracy concerning race track scanning for racing sims?
    Do we need some additional data for altitude ie. can we integrate GPS info to the scan?
    Are multiple cameras possible for simultaneus forward & backward scanning?
    Can you give any estimate when 2.0 will be out? this year, next year?

    Thank you , keep up this tremendous job!

    1. Hi,

      Accuracy can be good at short distance. Try PVS1 and see the results.
      The further we go (from the camera), more guessing and interpolation we get.
      It's also a really sensible system to good calibrations.
      But for large surfaces, uniform ones it's not easy to get features to track and it will be harder to get a good reconstruction. This might be the main problem with racetracks (modern ones) with large open areas).
      I hope to solve this with omnidirectional lenses.
      GPS will help for large areas/distances.
      PVS2 is all about multiple sources. Cameras and GPS (hopefully).
      Estimate for this year, maybe before summer ;)

    2. May be precission could be improved by adding another camera pair? lets put two cams on the roof and two cams at the heigh of the car lights, shouln´t this improve precission? may be not as 3d points are built only from image pairs without considering other pairs right?. This is something I did not like much from v1 package as if you put long computing distance it did calculate points far away with low precission that once you where closer to these points and your ecalculate these from closer pair of photos the more accurate ones messed with the less acurate ones.

  4. My main focus is to get the racetrack itself modeled with it's bumps and cambers properly. So for a 20m wide track would you suggest a double pass and the join the two meshes manually?

    What would you say the accuracy would be for track surface depthwise (using C920 cameras)?

    The "maybe before summer" sounds very _very_ good :)

    Good job man

  5. Hello,
    Scanner is one of the best invention of technologies for saving human important time. This machine is very useful. The way you post your article this is such a nice sharing that I have.

  6. Hey Pablo
    Any more update news?

  7. Please update>> visiting this in a regular basis and feels a bit too shy :\

  8. Hi! Any progress on this? Really hoping to try it someday!

  9. I check back on this site every now and again. But its been over a year since the last update! I sincerely hope this project is still alive. I'm sure a lot of people (myself included) are keen to use it. Any chance you could let us know how its going? Thank you for all your efforts to date.