Device Settings
From ColourWiki
All variable parts of the colour processing chain must be taken into account, to achieve predictable results. As a consequence colour profiles as colour only transformations are not enough to describe completely the path colours have to go. For instance downloading a printer profile and separating an image does not ensure the according driver settings are selected correctly or even the appropriate driver was used. Even the driver and its version is often not included and not standardised to access machine readable. To overcome that information hole for monitors, Apple introduced the vcgt tag to describe the according hardware state in the graphics card gamma table. This is a single step and maybe enough for just this task. But we should go further to make ICC profile selection rebust in a general context.
The following text will outline needed changes and to make the task complete.
Advantages
Usability Objectives:
- select an profile and the system/application selects the correct driver settings to the calibration state, for which the ICC profile was created
- change settings and the system detects if a profile is available for that calibration state
- email a ICC profile with identification and calibration data inside, no further information is needed to setup
- use the same profiles in all applications, they are globally available through the ICC profiles themselves
- exchange these ICC profiles through the web easily and by sure they work locally
- good out of the box experience
- user decisions have highest priority (manual selection)
- robust opt-out of colour management for experts (e.g. for ICC profile creators)
Profile Selection
We handle user and system configurations, explicit and automatic selection and fallback profiles in a well separated manner.
Order of profiles selection:
- explicit user settings - inside the users Oyranos DB
- explicit system settings - inside the systems Oyranos DB
- automatic selection from all available profiles - ICC profile DB
- query all profiles 'meta' tag and select the best match to a given device calibration
- driver module fallbacks
- e.g. ICC from EDID for oyX1 and qarz
- e.g. cupsICCProfile for CUPS
- The profile is generated on the fly of provided by other specific means of the module.
- Oyranos default colour space fallback
Explicit configured profiles shall not interfere with automatic profile selection. Automatic generated fallback profiles must be marked as such and are lower ranked than normal ICC profiles (TBD).
Scenario
(X example, similar to print, scan, digicams):
- obtain monitor settings informations
- set/reset monitor settings from software (xcalib/ddccontrol,...)
- interface to get/set settings from Oyranos to drivers
- extract ICC information easily
as Inkscape SVG (http://www.oyranos.org/wiki/images/d/da/Device_profiles_01.svg)
Informations Incorporated
- device identification
- device type, manufacturer, model, serial number, country, device options
- AB & Co Ltd., printer model CD-1234, EF photo ink, GH gloss paper, #1234-5678
- driver names and their versions to verify integrity
- nv/nvidia, Gutenprint/HPIJS, DCraw versions
- The driver names used must be unique to later lookup and find the appropriate profile. If one application use Gutenprint and an other gimp-print chances are high to not find the according profile. Of course some kind of registering probably with a enumeration type would help here. With a good maintained database this should work fine.
- The driver must be able to tell: down to what driver version he is colour compatible for a certain device. The versioning should be clear to the device module. Its likely to get specialised for each driver.
- device settings (as provided, colour relevant ones should be extracted by a device driver API)
- expl. PPD, scan parameters, VCGT
- fixing colour sensible driver settings in a GUI to exclude wrong user interaction is a precondition to use always the same calibration for the selected ICC profile.
- PPD with removed choices, VCGT + fixed monitor settings, DCraw called with specified options
- device colour characterisation (a ICC profile belonging to the above outlined calibration)
- the profile must belong to the device/driver/settings/version mix
- priority among selected profiles (generic profile - low, self calibrated - high) can be a profile property
- fuzzy device/driver settings match outlined by Graeme here (http://lists.freedesktop.org/archives/openicc/2005q2/000313.html)
- distribute by combining above information
- packing all informations into one piece
- embed into ICC profile, register in a local or remote hardware db, xml configuration
Driver
Device driver modules in Oyranos extract the colour related informations, serialise them into key/value text pairs and place the result into the Oyranos device object. The module provides a rank map, for how to interprete the key/value pairs. This will be device class and driver specific. The ICC profile is selected according to the device calibration state and rank map provided by the device class module.
Profiling
Implementation Details
- easy accessible text format to store and handle driver informations (key/value pairs) in the Oyranos DB
- driver identifier (which every application can either understand or reject the whole profile)
- possibly allow data blobs too, to allow compressed content
- Graeme Gill suggested to use an magic number to identify the blob.
- additional part within this format to store CMM specific thing (expire date, ...)
Overview of possible approaches to ICC profile with calibration data bundling:
- add device settings information to an existing ICC profile
- implemented in Oyranos through the ICC 'meta' tag
- (see oyranos-monitor tool)
- use a internal dedicated ICC profile tag
- possibly hard to circumvent license issues
- best to identify, without breaking ICC standards
- allow profilers do handle themselves (by an small library)
- need possibility to add that information after profiling, because the device informations may not be available at profiling time or the profiler is simply not aware of our demand
- check and correct the size tag in ICC profiles and append the device settings text behind. The profile size would be the offset to that data.
- profile readers and CMMs would handle as usual
- non standard
- someone else can use the same approach, which would make such data useless
- give the modified a proper extension to identify; ?? ICG for instance (InterColorprofileGathering) ??
- prepend the text and define an offset to the ICC profile data
- breaks existing CMMs and profile readers
- clear to handle, to be covered by an specification
- forces a switch first
- use a internal dedicated ICC profile tag
- fuzzy matching should be possible see here
- the access algorithm description (http://lists.freedesktop.org/archives/openicc/2011q1/003036.html) for the Oyranos DB
Oyranos API suggestion
/* sequence to get a device profile */ oyDevice_s * device = 0; oyProfile_s * profile = 0;
oyDeviceGet( "config", "camera.dcraw", "absorber-2.0", 0, &device ); /* depending on the device class add the following */ /*oyDeviceSetDriverContext( device, (void*) driver_context_pointer, (const char*) "string.dcraw.xml", (size_t)0 );*/ oyDeviceGetProfile( device, &profile );
Specification
| Device Settings in ICC Revision History | |||
|---|---|---|---|
| v0.1 | 2006-08-28 - 09-04 | Kai-Uwe Behrmann © 2006 | |
| v0.2 | 2006-09-06 | Kai-Uwe Behrmann © 2006 | Thanks for suggestions and revisions to Graeme Gill, Hal V. Engel, Robert L Krawitz and Gerhard Fuernkranz. |
TODO (v0.3):
- The Priority field should be limited to one profiler. Otherwise its value is object for concurrency, which is not desirable.
- change the name to 'drvS' to tell its dedication is about driver settings not device resolving
- remove the device resolving and hint to use the 'pddt' tag instead
- rename the Device Settings in ICC to Driver Settings in ICC
The ICC meta tag provides a more generic framework for device and driver informations. This can be easily deployed after serialisation into key/value pairs, for device identification and driver settings for device calibration.
OpenICC Configuration 0.1 specification
Specific Considerations
Printing
Architectural Design
- adhering to existing standards
- ensure interoperability
- if in trouble, simply file a bug for non ISO standard behaviour
- non redundant ICC transport
- avoid double transportation of ICC informations
- no ICC information transport outside the PDF itself
- disambiguate user decisions (various systems may conflict)
- better traceability in case something goes wrong
- fewer points of fail
- better predictability of expected results
- selfcontainment
- PDF is the Linux print spool format
- all colour management information can reside inside PDF
- rely for information from CUPS to a local host on exposed PPD
- ColorKeyWords can help with easier user profile assigment
- clear priorities for ICC profile selection
1. User assigned profiles (User is King) 2. System assigned profiles 3. Automatic resolved ICC profiles (from local sources or internet) 4. Server side fall back (canned vendor ICC profiles) 5. fallback to defaults Case 4 will be the most often used followed by 3 and 1. Case 5 is to be overcome but still in place.
- non ambiguous ICC device profile to device calibration assignment
- device identification and colour related driver settings can not be split
- changing driver single channel brightness makes the ICC profile unreliable - do not allow that
- device identification and colour related driver settings can not be split
CUPS
Preparation of local spool PDF:
Inkscape drawing (http://www.oyranos.org/wiki/images/OyranosCMS-2_devices-printing-preparation03.svg)
ICC profile selection for printing devices with Oyranos:
Inkscape drawing (http://www.oyranos.org/wiki/images/OyranosCMS-2_devices-printing-selection_03.svg)
- CPD Presets name in ICC (http://wiki.openusability.org/wiki/printing/index.php/Dialog_Levels#Level_2_-_Layout_Principals) as designed in the CPD usability study. The "APPrinterPreset" (http://www.linuxfoundation.org/collaborate/workgroups/openprinting/ppdextensions#Presets) key (http://lists.freedesktop.org/archives/openicc/2011q2/004229.html) can reside with the calibration inside the ICC meta tag.
Details:
The proclaimed
wget http://localhost:631/profiles/sRGB.icc
is not working in CUPS(v1.2), as pointed out by Hal V. Engel and confirmed (http://www.cups.org/str.php?L2894) by Mike Sweet.
A path to send a user selected local device profile to a remote print host is unclear, at least to me.
The limitation of one CUPS profile directory, e.g. "/usr/share/cups/profiles", is too restrictive, even with linking to other directories. The cupsICCProfile attribute should therefore allow for absolute locations, not just relative to a CUPS profile directory. The CUPS_DATADIR seems not a appropriate mechanism.
CUPS passes data from the data directory, e.g. the web documentation pages. It is possible to link the CUPS profile directory to that and the above mentioned wget http://localhost:631/profiles/sRGB.icc should work. Open is, how to know the data_root path. Much related is Till Kampeter's support request for by default exposing ICC profiles from PPDs on the CUPS web server (http://www.cups.org/str.php?L2896).
- /usr/share/cups/profile/ is not a standard search path for profiles on Linux (http://www.cups.org/str.php?L3793) bug report
A print job should reflect the users local settings in regards of rendering intent selection, editing space and so on.
How are profiles, rendering intents and other settings passed from a printing application to backends, e.g. the *toraster family. Do the RIP's need to parse the PPD in order to read the cupsICCProfile PPD attribute?
- use a job ticket format as discussed on OpenICC
How copy locally stored profiles to remote printing hosts?
- use the (http://lists.freedesktop.org/archives/openicc/2011q1/002769.html) PDF OutputIntent (http://lists.freedesktop.org/archives/openicc/2011q1/002770.html) to embed a print device profile
- osX uses the OutputIntent (http://lists.freedesktop.org/archives/openicc/2011q2/004075.html) to ensure transportation of local ICC device profile to remote location
Linux will switch to PDF (http://www.linuxfoundation.org/en/OpenPrinting/PDF_as_Standard_Print_Job_Format) as standard format for printing. The advantage of Ghostscript would be to obtain PDF colour management (https://www.linuxfoundation.org/en/OpenPrinting/Color_Management) right from the beginning.
Rasteriser:
Poppler is colour management enabled and has a pdftoraster filter.
see Ghostscript to build a test workflow
Hints
- switch between Poppler and Ghostscript (http://lists.freedesktop.org/archives/openicc/2011q1/002842.html)
Issues
- [fixed] Ghostscript has a not working pdftoraster (http://bugs.ghostscript.com/show_bug.cgi?id=690101) filter.
- Ghostscript PDF rendering intent not honoured (http://bugs.ghostscript.com/show_bug.cgi?id=691952) bug entry
- Poppler ignores rendering intents and always uses INTENT_RELATIVE_COLORIMETRIC (https://bugs.freedesktop.org/show_bug.cgi?id=34053) bug entry
- Rendered output is currently restricted to 8 bit/channel (https://bugs.freedesktop.org/show_bug.cgi?id=34055)
- CPD and Color Management (http://lists.freedesktop.org/archives/openicc/2011q2/003595.html) thread shows how to use the Gutenprint API to convert native settings to and from XML
Desirable APIs for Oyranos
- obtain a profile by giving Oyranos a static PPD (one with all open choices stripped).
- obtain a list of profile/static_PPD combinations for a specific print queue.
Solutions for Color-Managed Printing
Profile selection and PDF rendering capability for printers is being provided from the following projects:
- XCPD (eXperimental Common Printing Dialog) - A Google Summer of Code 2011 project that models color management for the CPD.
- libCmpx (Library for Color-Managed Printing eXtension) - Library derived from the XCPD project. It will provide a general mechanism to extend color management to printer dialogs.
Calibration and Profiling
- Calibration and separation levels in ArgyllCMS (http://lists.freedesktop.org/archives/openicc/2011q2/003659.html) from Graeme Gill
- Setup, Calibration and color Profiling State (http://www.argyllcms.com/SCPS_Tag.txt) information tag for print files from Graeme Gill
- calibration state passing the last mile (http://lists.freedesktop.org/archives/openicc/2011q2/004212.html) from Hal V. Engel
- Ghostscript PDF/X-3 (http://lists.freedesktop.org/archives/openicc/2012q2/004851.html) handling of DeviceXXX/ as needed for linux printing filters
References
- PDF colour management wiki page (https://www.linuxfoundation.org/en/OpenPrinting/Color_Management) @linuxfoundation/OpenPrinting
- CUPS and ICC colour management email (http://lists.freedesktop.org/archives/openicc/2005q2/000227.html) thread (http://lists.freedesktop.org/archives/openicc/2005q2/000208.html) on OpenICC
- CUPS on linuxfoundation.org (http://linuxfoundation.org/en/CUPS)
- v4 ICC support request for Ghostscript #688036 (bugs.ghostscript.com) (http://bugs.ghostscript.com/show_bug.cgi?id=688036)
- not working pdftoraster bug #690101 (bugs.ghostscript.com) (http://bugs.ghostscript.com/show_bug.cgi?id=690101) in Ghostscript
- Poppler CM request #17499 (bugs.fd.o) (https://bugs.freedesktop.org/show_bug.cgi?id=17499)
- Poppler display patch set (http://lists.freedesktop.org/archives/openicc/2011q1/002795.html) posted by Hal V. Engel
- OpenPrinting CM page (https://www.linuxfoundation.org/en/OpenPrinting/Color_Management) @ linuxfoundation
- Colour marking of PPD options (http://www.freedesktop.org/wiki/OpenIcc#PPDcolouring) from Kai-Uwe Behrmann
- PDF Rendering Pipeline.pdf (http://www.box.net/shared/o0h267hk3d) from Adobe
- LINUX color management a proposal for implementation (http://www.freedesktop.org/wiki/OpenIcc#LINUXcolormanagementaproposalforimplementation) from Jan-Peter Homann
- Add a device-inhibit command to the colormgr client (https://gitorious.org/colord/master/commit/fe07880032d6c74d93495e57347709a856e7fa9d) skip profile selection through colord CUPS hook
Camera Raw Developing
- DNG (http://www.adobe.com/products/dng)
...
Scanning
- tagging colour management patches (https://alioth.debian.org/tracker/index.php?func=detail&aid=312641&group_id=30186&atid=410366) bug entry for SANE
Displaying
Oyranos/X11 Requirements is the page to collect all informations
Compiz plugin based ICC colour server (http://compicc.sf.net/)
ICC meta Tag for Monitor Profiles 0.1
Wayland Color Management Proposal (http://people.collabora.com/~pq/wayland-color-management-proposal.png)
Wayland Color Management gist file (https://gist.github.com/Zoxc/3681678)
Weston CMS protocol (https://github.com/Zoxc/weston/blob/cms/protocol/cms.xml)
Configuration
- Kolor-manager
- configuration dialog of Oyranos - plans to use XMLised GUI
- oyranos-monitor, ... - command line tools
Module Protocol
Oyranos core <-> device module protocol
Implementation Overview
Oyranos implements devices as oyConfig_s objects.
A 'oyConfig_s' object holds a set of options coming from a device backend and possibly from the Oyranos DataBase (Oyranos DB).
Options which are relevant for inclusion into the Oyranos DB must be build of text strings. These are the oyConfig_s::backend_core options.
The device options represent both device attributes and driver settings in one object.
The device backend is implemented in Oyranos with the oyCMMapi8_s structure. This structure must be included in a module and be pointed to from a oyCMMInfo_s structure like all modules.
Tables
Options marked with "in:" are passed through a oyOptions_s argument to the module. Outgoing options are passed through the device(s) created by the modules and are assigned as follows:
[c] - oyConfig_s::backend_core - identification stuff as strings
[d] - oyConfig_s::data - additional data
[b] - oyConfig_s::db - only for Oyranos core and thus tabu for modules
Required options are marked with a [r].
The "device_name" option shall only be considered in oyConfig_s::oyConfigs_FromPattern. In oyConfig_s::oyConfigs_Modify the oyConfig_s's devices should have already that property in their backend_core part.
| Property | Modules | Informations | Data format |
|---|---|---|---|
| "command"="list" | oyX1, SANE, CUPS, oyRE | in[r]: call for detected device(s) | in: string |
| "device_name" | oyX1, SANE, CUPS oyRE | device ID in: select a single device; out[r]: device ID any meaning for oyRE? | in(oyConfigs_FromPattern)/out[c]: string |
| "oyNAME_NAME" | oyX1, SANE, CUPS, oyRE | in: call for lightweight informations out: short UI description | in/out[d]: string |
| "icc_profile" | oyX1, CUPS | out: the ICC profile accessible through the device driver | in: string; out[d]: oyProfile_s or a empty "icc_profile" option of oyVAL_STRUCT, if ICC profiles are supported by the device but in the actual search not found. |
| "display_name" | oyX1 | in: select a X11 display and get all connected devices; "device_name" has priority | in(oyConfigs_FromPattern): string |
| "device_rectangle" | oyX1 | out[r]: the device rectangle in pixels | in: string out[d]: oyRectangle_s |
| "supported_devices" | oyRE | out: one manufacturer in the first line, remaining lines each for one model | in: string out[d]: strings |
| "device_context" | oyX1, SANE, CUPS, oyRE | out: not required "SANE_Device" "ppd_file_t*" "libraw_output_params_t*" | out[d]: string |
| device handles | SANE, CUPS, oyRE | version + SANE_Device + SANE_Handle out: device handle caching | out[c]: oyCMMptr_s or oyOption_SetFromData() but no string(s) |
Note: If a "manufacturer" or a "model" option appear in a device after the "list" call, the device is considered ready for a Oyranos DB query. Thus all options which normally appear after a "properties" call should be included together with "manufacturer" or "model" options.
| Property | Modules | Informations | Data format |
|---|---|---|---|
| "command"="properties" | oyX1, SANE, CUPS, oyRE | in[r]: call for even expensive informations | in: string |
| "device_name" | oyX1, SANE, CUPS, oyRE | in: select a single device; out: ID of each device, e.g. <manufacturer>-<model> | in(oyConfigs_FromPattern)/out[c]: string |
| "display_name" | oyX1 | in: select a X11 display and get all connected devices; "device_name" has priority | in(oyConfigs_FromPattern): string |
| "manufacturer" | oyX1, SANE, CUPS, oyRE | out[r]: device manufacturer | out[c]: string |
| "model" | oyX1, SANE, CUPS, oyRE | out[r]: device model | out[c]: string |
| "serial" | oyX1, SANE, CUPS, oyRE | out: device serial number | out[c]: string |
| "system_port" | oyX1, SANE, CUPS, oyRE | out: X11 screen number out: scsi/usb/parallel port? out: print queue? out: RAW file name? | out[c]: string |
| "host" | oyX1, SANE, CUPS, oyRE | out: X11 server host name out: possible?t out: print host possible? out: "local" host? | out[c]: string |
| "display_geometry" | oyX1 | out[r]: X11 screen geometry as in "device_rectangle" | out[c]: string |
| "edid" | oyX1 | out: monitor EDID block | in: string out[d]: use oyOption_SetFromData(); users - oyOption_GetData() |
| "device_context" | oyX1, SANE, CUPS, oyRE | in: not required SANE_Device ppd_file_t*? libraw_output_params_t* | in: add with oyOption_SetFromData() |
| device handles | SANE, CUPS, oyRE | in: abbreviation to talk with drivers | in[c]: oyOption_GetData() |
| device settings | oyX1, SANE, CUPS, oyRE | out[r]: "display_geometry" ... ... static colour related PPD attributes libraw options | out[c]: string |
| Property | Modules | Informations | Data format |
|---|---|---|---|
| "command"="setup" | oyX1, SANE, CUPS, oyRE | in[r]: call for uploading a ICC profile through the device driver do nothing set a cupsICCProfile attribute in a given PPD and write out to file do nothing | in: string |
| "device_name" | oyX1, SANE, CUPS | in[r]: select a single device | in(oyConfigs_FromPattern): string |
| "profile_name" | oyX1, SANE, CUPS | in[r]: the local accessible ICC profile file name | in: string |
| "config_file" | CUPS | in: a PPD configuration file (ppd_file_t*) to modify or nothing; out: (ppd_file_t*) | in: oyOption_GetData(); out[d]: oyOption_SetFromData()? |
| device handles | SANE, CUPS, oyRE | in: abbreviation to talk with drivers | in[c]: oyOption_GetData() |
| Property | Modules | Informations | Data format |
|---|---|---|---|
| "command"="unset" | oyX1, SANE, CUPS, oyRE | in[r]: unloading the _ICC_PROFILE_xxx atom do nothing remove a cupsICCProfile attribute in a given PPD do nothing | in: string |
| "device_name" | oyX1, SANE, CUPS | in[r]: select a single device | in(oyConfigs_FromPattern): string |
| "profile_name" | oyX1, SANE, CUPS | in[r]: the local accessible ICC profile file name | in: string |
| "config_file" | CUPS | in: a PPD configuration file (ppd_file_t*) to modify or nothing; out: (ppd_file_t*) | in: oyOption_GetData(); out[d]: oyOption_SetFromData()? |
| device handles | SANE, CUPS, oyRE | in: abreviation to talk with drivers | in[c]: oyOption_GetData() |
| Property | Modules | Informations | Data format |
|---|---|---|---|
| "command"="help" | oyX1, SANE, CUPS, oyRE | in[r]: show help text through the modules local oyMessage_f | in: string |
Calling a device module without any command or a unknown command shall issue a message through the modules local oyMessage_f. The message shall contain infos about, describing calls and their properties and the expected results. Otherwise adhering to the above protocol will ensure a base level of interoperability with users like KM.
How to create a useful "device_name" (ID)?
it should be unique in that the module can distinguish each device and understand to build a device context. This is important for the case when no device context pointer/handle is passed from the user. How that is defined is up to the module writer. Even though, a string, one which makes sense in KM, would be much appreciated. e.g. in oyX1 it is "device_name"=":0.1" or "device_name"="far-away.org:10.0". The modules XOpenDisplay() can directly use these strings and the module can create a context and talk to the device. For SANE you have choosen "pnm:0", "pnm:1" and here locally I see "plustek:libusb:003:002". The last is unfortunately subject to change with a new usb port or after newly plugging in. But that seems acceptable.
Handling device contexts
The "device_context" option can be set from the module during the "list" call into a device. The user can then see, which specific information is accepted by the module. The module has implicitly to assume, that the user has no background about the called device class, and to initialise a device by its own. In case the user has already a context and passes this one to Oyranos, the module needs to extract the colour related device settings from this and pass the device settings as string to Oyranos. Oyranos will then be able to search for a according ICC profile.
It is important for Oyranos, the module and the user to agree who is responsible for the driver contexts. oyOption_SetFromData() is useful to pass a pointer through a oyOption_s. The size argument can be set to zero to avoid freeing the pointer through Oyranos. In case the module provides a context, e.g. in "list" then that needs to be released through driver functions. Oyranos has the oyCMMptr_s struct defined in oyranos_cmm.h which can be used: [1] (http://www.oyranos.org/doc_alpha/structoyCMMptr__s.html).
Multiple Configurations per "device_name"
For backends its possible to resolve one "device_name" option to multiple driver configurations. Such backends are best implemented in that they resolve the oyConfig_s objects right from the beginning, e.g. the "list" call. Otherwise the oyConfig_s devices are not uniquely defined. On a practical example, a CUPS device would be uniquely defined, if all colour related and to be included PPD entries are included in the oyConfig_s::backend_core options as strings.
Users can subsummarise such devices in that they look at the registration, manufacturer, model and serial number strings. However they should make clear that each device consists as well of the driver settings.
Further Reading
The above protocol is to be implemented in Oyranos device modules. See: [2] (http://www.oyranos.org/doc_alpha/extending_oyranos.html#write_device_modules)
