[MagicLantern] raw2cdng – Cinema DNG

i thought, cinema dng could be the right format to open the rawdata into the bigger grading world, for example speedgrade and resolve. now we have to see. for now there is this tool for windows and the cdng-converter rawmagic for mac written by thomas worth. my github link was only for first discussion purposes. you will find the CinemaDNG-Dialog in the Magiclantern-Forum.
Resolve_16Bit_magiclantern_cinemadng

important!

– its just a raw to cinema dng converter – its NOT debayering!
– needs .net 4.0 Runtime – here
– made for windows-systems – not tested under emulators/virtualizers.
– its coded from scratch and has nearly nothing to do with raw2dng!
– this is a beta! need your response!


new Version 1.5.0 is in a new blog entry

blog.phreekz.de raw2cdng 1.5.0 magiclantern


raw2cdng10


version log and informations

11.06.2013 – v1.0 – fixes so far
– batchmode integrated
– blacklevel, whitelevel, colormatrix read from rawdata
– assuming body by first colormatrix-value
– 12bit linear built in (lsb 2bit cropped)
– 2gb crash fixed
– lowering whitelevel by 15% – maybe a helper for pink highlights
– tried to fix the slash/backslashproblem for pathes

11.06.2013 – v1.1.2 – fastfix
– if blacklevel wasn’t found in raw – it turned zero – fixed that to 2037
– blacklevel for 12bit was too high (/2 -> /4)
– stripByteCounts calculated now from resolution and bitsperPixel – reading the raw from newer ml-builds

14.06.2013 – v1.1.6 – possibly EDL/XML fix
– inserted Reelname-Tag with 8 Chars
– changed filename of pictursequence to section filename and sequencenumber with a dot.
– allowed for filename are only a-z and 0-9, other symbols will be deleted

20.06.2013 – v1.1.8 – 2nd attempt – EDL/XML fix
– put framerate and timecode into cdng-header
– reelname fix – nothing happend if filename is shorter than 8 chars

RESOLVE ROUNDTRIP IS WORKIN !
23.06.2013 (!) 1.1.8 was not the solution for functional roundtrip BUT peoplemerge found, how to solve the problem – you have to fill up the ReelNo-Attribute in Resolve to get unique Filenames for xml/edl-roundtrippin‘ – please look into his video at ~7:10

27.06.2013 – v1.1.9 – 10bit log
– 10bit log (!) experimental – first trials (red tint, but adjustable with red gain to ~0.86)
– pathnaming -> timedate of creation -> [yymmdd_HHmm]_[REELNAME]
– filenaming -> [REELNAME].[sequenceNo].dng
– shortened filenames (M12-1305.raw) randomly filled up to 8 symbols f.e. M121305T
– fixed GUI-freeze – worker-threads

(!) 10bit log only works for bodies with blacklevel higher 2000 (!)
next patch will have a lut for blacklevels around 1024 (!)

Advantages of less bitdepth (on 1280*544px)
– 16bit (filled up 14bit) – 1,37MB per frame
– 12bit (2bit shifting) – 1,02MB per frame (-26%)
– 10bit/log (lut) – 0,86MB per frame (-38%)

30.06.2013 – v1.2.0 – splitted Files
– splitted Files support (you only have to drag the raw-file)
– splitted Files – please reply if it’s working flawless.. 🙂
– import-bug fixed – if path/filename was shorter than 23chars it didnt imported file

01.07.2013 – v1.2.1 – selectable static outputPath
– you can select a outputfolder
– means, you can transcode straight from the card
– the settingsFile is json-serialized saved in user/userX/appdata/roaming
– randomly added chars are now calculated from the existent name -> unique name

13.07.2013 – resolve 9.1.5 supports 14bit-dng
– i’m kind of pissed. instead of optimizing their debayer-modul (pink fringes all over the world), seriously reading/using the Reelname for xml, doin‘ some badpixel-support, noisereduction-tool or manual devignetting-sliders they have done the simplest thing. ok, its a free software – and i hope, they build this all into the newer v10-version – but thomas worth, a1ex, g3gg0, iliasG, fatpig and lots of other coders did a way better (free) work to make a handy environment to compete the ML_RAW-revolution. I#m now a little bit tired thinking about optimizations – by now i’m doin a pause – rethinking this tool – there are three things: simplifying convert-process, lowering datasize and opening as a commandline-tool. Ahh, forgot one challenge: accurate colors/whitebalance/colormatrices/LUT – we re talking about it in the ml-forum – in Speedgrade the problem is obvious.

11.08.2013 – v1.3.0 – whitebalance from cr2
– as by now i wont maintain raw2cdng. all things happening from now on are tests of functionality for the forthcoming mlv2cdng. all opinions and experiences are welcome, please feel free to write them here into the blog or into the magiclantern-thread
– GUI slightly changed
– this version’s got CR2-whitebalance-support. do a photo right before the movie record and put this one into the same path as the raw-file. name it as the rawfile. raw2cdng will recognise it automatically and read the RGGBValues asMeasured (from makernotes) and turn it into the whitebalance-data for the dng-files. ITS ONLY TESTED WITH 5DIII CR2-Files!

Example:
M02-1810.RAW
M02-1810.CR2

If you recorded a whole set of same whitebalanced stuff, you name this file ALL.CR2 and all RAWs will get this whitebalance-value – except files havint their own CR2.

Example:
M02-1631.RAW (WB from ALL.CR2)
M02-1635.RAW (WB from ALL.CR2)
M02-1810.RAW (WB from the according CR2)
M02-1810.CR2
M02-2012.RAW (WB from ALL.CR2)
ALL.CR2

09.11.2013 – v1.4.0 beta – experimental
This one’s is a first usability-test for photoshop, speedgradeCC and resolve10 users. by 90% stripping wont work. multithreaded. lowering blacklevel by 90%-button. changeable UniqueModelID for ACR-Profiles. framerate fixed. .look-file inside for speedgradeCC.

24.11.2013 – v1.4.5 – multithreading and mlv
* fixed modell-tag
* multithreading working with stripped files (tested with 1 strip)
* whitelevel back to static 15.000 (16bit)
* speedgrade-look – changed multiplicator to 16
* mlv support (not native, but via mlv_dump)

put mlv_dump.exe into the same path as raw2cdng. it will be recognised on start and shown in the title. now you can drag&drop mlv-files. they will be rewritten to raw after dropping. raw-files will be saved in the same path as mlv_dump.exe

24.11.2013 – v1.4.6 – empty modelID-Crash fixed
* crashed, when modelID-Field was checked but empty
* switchable debug-Log (saved in according .log-File in same path)

raw2cdng description

Since 1.4.8:
For ACR-Applications (AE,Photoshop), Speedgrade and Resolve please use
*(5) 16bit linear
*(6) no partymode, Gamma 1.0
*(4) enable and fill in the according Model ID for right profiles. Example: Canon EOS 5D Mark III
*(+) for Speedgrade you have to use a calibration-lut, there’s an example in v1.4.8

For Premiere CC (works with Resolve as well):
*(5) 12bit linear
*(6) partymode enabled, Gamma 1.0
*(4) Model ID not needed.

raw2cdng.1.4.8
04.12.2013 – v1.4.8 – partyMode and gamma
* to be honest, lets make party, because its the first step using cdng files straight inside premiere CC7.1
* set/change gamma to 1.0 – its built in for testing purposes.
* filename-prefixes are back on dng-files
* possibly stripped-files-bug fixed (didnt checked it with more than 3 strips)
* adjusted .look for speedgrade to new filedata
* inside premiere give the cineon-converter a try

09.02.2014 – v1.4.9 – audio from mlv
* audio recreation from mlv-files
* copied and renamed into Output-Footage-path
* in premiere just import the dng-sequence, the audiofile will be recognised automatically
* newest mlv_dump.exe from g3gg0 is inside the zip

19.02.2014 – v1.4.9.3 – mlv_dump fix
* wrong characters in path or filename leads to failure. fixed.
* modeltag and uniquemodel are separated (@Andy600 told about problems in cinelog)
* optimizing some things (fi tiff-dng-tag-creation)

12.03.2014 – v1.4.9.5 – native mlv with audio
* warning. test-version. tell all problems, bugs

16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed

21.03.2014 – v1.4.9.7 – pink highlights
* this is a testversion, if pinkhighlight-fix works with any body
* logarithmic approaches freezed
* partymode is now on by default, no click.
* raw-bug fixed (fps.denominator was the bad boy)


raw2cdng

download

v0.93 – not functional preversion for discussion
02.06.2013 – v0.99 – initial working version
11.06.2013 – v1.0 – see fixeslist above
11.06.2013 – v1.1.1 – see fixeslist above
11.06.2013 – v1.1.2 – see above
14.06.2013 – v1.1.6 – attempt to fix xml/edl
20.06.2013 – v1.1.8 – 2nd attempt to fix xml/edl
27.06.2013 – v1.1.9 – see above
30.06.2013 – v1.2.0 – splitted Files
01.07.2013 – v1.2.1 – output Folder
11.08.2013 – v1.3.0 – CR2-whitebalance
09.11.2013 – v1.4.0 beta – experimental multithreaded
24.11.2013 – v1.4.5 – mlv – speedgrade-look inside
24.11.2013 – v1.4.6 – empty modelID-Fix
04.12.2013 – v1.4.8 – partymode
09.02.2014 – v1.4.9 – audio
19.02.2014 – v1.4.9.3 – mlv_dump fixed
12.03.2014 – v1.4.9.5 – native mlv-support
16.03.2014 – v1.4.9.6 – bugfixing mlv-code
21.03.2014 – v1.4.9.7 – pink highlights

version 1.5.0 is on new blogentry!











Dont forget to donate the Magiclantern team (and maybe then me :)) This button is for donations for my job. But, ML did a fabulous job on enabling this hell of a gem! regards chmee


ToDo

– vertical lines banding (solved in raw2dng, but have to implement it)
– compression (important or not?)

done!

audio from mlv
threading in conversion – speeding up the thing.
static outputfolder
splitted files (raw r00 r01 etc pp) support
fix 2gb crash
embedding 12bit linear as well
embedding other color matrizes than 5DIII
– pink highlights (problem longer known, dcraw user saying, lowering the saturationlevel is the key. satlevel=whitelevel? with ETTR enabled in Magiglantern it shouldnt be no longer a problem)
there was a change in raw.c code – the stripbytecounts changed – have to fix that. (urgent!)
whitebalace refining via CR2-makernotes v1.3.0

ACR_CinemaDNG_magiclantern_raw

62 Kommentare

  1. thank a lot. i just prepared the code for some preprocessing-code. so vertical banding and chroma smoothing could be inside 1.5 🙂 BUT its depending on the clarification of the ml-team (a1ex?) about their GPL-License and using in my closed-source. we’ll see..

  2. Hi chmee,
    I am having trouble with raw2cdng.

    Basically it crashes a lot, particularly when batch processing. I’m running it on a brand new fast PC and have had no problems with other programs such as RAWanizer etc.

    I’ve tried several versions, although the latest two 1.5 betas won’t open at all.

    Typically it will load the files just fine and let me select my settings, output etc. But when I click Start it crashes.

    I’m desperate to use Resolve as part of my workflow. Any ideas anyone?

  3. Dylan, please write things regarding versions =>1.5 into the according blog-entry 🙂 please be patient, to solve a lot of things i have to refine the debug-log (i’m already coding that) – i hope i will give a beta4 after the weekend..

  4. Thanks chmee,
    I just tried the new beta and had no trouble. I think raw2cdng was crashing because the drive I was creating to wasn’t fast enough.

  5. sir,

    Thanks a lot for your work, from 5d mark 3 mlv raw 4gb to 32 gb (single file) can we get dng files out of it?.

    regards
    Madan jagannatha

  6. Hello . First, about outstanding work! My question: Is it possible to set white balance and brightness ? Unfortunately, I recognize the proxy images almost nothing. The properties are deleted. ( Aperture 0 , etc.)

    Sorry for my bad English . Thank you for the great work !!

  7. Patrick, german maybe?
    So, no, its not intended, but possible. yes, the properties are deleted in a 8bit-jpg or mp4. maybe there’s a bug inside my tool, if the proxy-pictures are too dark. send an example to raw2cdng@phreekz.de or talk in the ml-forum with me.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert