Accessing Image Region Metadata in Node.js

An image file (e.g. JPEG, PNG) contains metadata, i.e. information about the image, e.g. which camera model was used or when the picture has been taken. This information is usually stored at the beginning of the file. Three main formats, or bags of metadata, can coexist in a file and the information they contain partly overlap:

IPTC IIM, developed in the early 1990s;
Exif, late 1990s;
XMP, late 2000s.

The IPTC IIM format (often just called IPTC format) and the Exif format represent sets of key-value pairs, whereas the newer XMP format is an XML representation of a more complex RDF graph. The XMP Specification Part 3 specifies how the XMP metadata are to be serialized and stored in each image file format (e.g. JPEG, PNG).

The IPTC council has defined a standard for storing Image Regions in XMP. Image Regions are useful for describing specific areas of the image (e.g. objects, people) or for indicating how the image should be cropped or rotated to best fit a given container. The Frameright app can be used to define such Image Regions and insert them in the metadata of a picture.

This tutorial shows how to read the Image Region metadata of an image with Node.js. This is especially useful if you want to access image metadata from within a backend application implemented using the Node.js. You can also use a bundler (e.g. Webpack or Rollup) to run what is demonstrated in this tutorial from inside a browser, which is useful if you want to access image metadata from within the frontend part of your application.

Setting up the environment

Let’s install Node and NPM. On Ubuntu 22.04 for example these tools can be installed with:

curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
sudo apt-get install -y nodejs

These tools are also available on macOS and Windows. The rest of this tutorial assumes you are running a bash terminal.

Setting up the project

For parsing image metadata we will make use of the Frameright/image-display-control-metadata-parser library, which is mainly a wrapper around the ExifReader library.

The IPTC Photo Metadata Standard 2021.1 Reference Image file contains several recently-defined XMP metadata items including Image Regions. Let’s download it with:

mkdir -p /tmp/frameright
cd /tmp/frameright
curl -O \
  https://iptc.org/std/photometadata/examples/IPTC-PhotometadataRef-Std2021.1.jpg

Let’s pull the library inside our project with:

npm install @frameright/image-display-control-metadata-parser

Accessing the XMP metadata

Let’s start first by extracting and dumping all the XMP metadata. Let’s create a Node script:

#!/usr/bin/env node
// /tmp/frameright/myscript.mjs

import { promises as fs } from 'fs';

// load the pulled library:
import { Parser } from '@frameright/image-display-control-metadata-parser';

const buffer = await fs.readFile('IPTC-PhotometadataRef-Std2021.1.jpg');
const parser = new Parser(buffer);
const xmpMetadata = parser.getXmpMetadata()

// dump the XMP metadata as XML:
console.log(xmpMetadata._raw);

We can now make it executable and run it with:

chmod a+x myscript.mjs
./myscript.mjs

Simplified output:

<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='Image::ExifTool 12.41'>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

 <rdf:Description rdf:about=''
  xmlns:Iptc4xmpCore='http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/'>
  <!-- ... -->
 </rdf:Description>

 <rdf:Description rdf:about=''
  xmlns:Iptc4xmpExt='http://iptc.org/std/Iptc4xmpExt/2008-02-29/'
  xmlns:exif='http://ns.adobe.com/exif/1.0/'
  xmlns:xmp='http://ns.adobe.com/xap/1.0/'>
  <Iptc4xmpExt:ImageRegion>
   <rdf:Bag>
    <rdf:li rdf:parseType='Resource'>
     <Iptc4xmpExt:RegionBoundary rdf:parseType='Resource'>
      <Iptc4xmpExt:rbH>0.385</Iptc4xmpExt:rbH>
      <Iptc4xmpExt:rbShape>rectangle</Iptc4xmpExt:rbShape>
      <Iptc4xmpExt:rbUnit>relative</Iptc4xmpExt:rbUnit>
      <Iptc4xmpExt:rbW>0.127</Iptc4xmpExt:rbW>
      <Iptc4xmpExt:rbX>0.31</Iptc4xmpExt:rbX>
      <Iptc4xmpExt:rbY>0.18</Iptc4xmpExt:rbY>
     </Iptc4xmpExt:RegionBoundary>
     <Iptc4xmpExt:rRole>
      <rdf:Bag>
       <rdf:li rdf:parseType='Resource'>
        <Iptc4xmpExt:Name>
         <rdf:Alt>
          <rdf:li xml:lang='x-default'>Region Boundary Content Role Name (ref2021.1)</rdf:li>
         </rdf:Alt>
        </Iptc4xmpExt:Name>
        <xmp:Identifier>
         <rdf:Bag>
          <rdf:li>https://example.org/rrole/role2021.1a</rdf:li>
          <rdf:li>https://example.org/rrole/role2021.1b</rdf:li>
         </rdf:Bag>
        </xmp:Identifier>
       </rdf:li>
      </rdf:Bag>
     </Iptc4xmpExt:rRole>
    </rdf:li>
   </rdf:Bag>
  </Iptc4xmpExt:ImageRegion>
 </rdf:Description>
</rdf:RDF>
</x:xmpmeta>

Let’s now adapt our script to only parse the Image Regions:

// ...
const xmpMetadata = parser.getXmpMetadata()

console.dir(xmpMetadata.ImageRegion.value, { depth: null });

Running the script now gives us (simplified output):

[
  {
    Name: {
      description: 'Listener 1'
    },
    RegionBoundary: {
      description: 'rbH: 0.385; rbShape: rectangle; rbUnit: relative; rbW: 0.127; rbX: 0.31; rbY: 0.18'
    },
    rId: { value: 'persltr2', attributes: {}, description: 'persltr2' },
  },
  {
    Name: {
      description: 'Listener 2'
    },
    RegionBoundary: {
      description: 'rbRx: 0.068; rbShape: circle; rbUnit: relative; rbX: 0.59; rbY: 0.426'
    },
    rId: { value: 'persltr3', attributes: {}, description: 'persltr3' },
  },
  {
    Name: {
      description: 'Speaker 1'
    },
    RegionBoundary: {
      description: 'rbShape: polygon; rbUnit: relative; rbVertices: rbX: 0.05; rbY: 0.713, rbX: 0.148; rbY: 0.041, rbX: 0.375; rbY: 0.863'
    },
    rId: { value: 'persltr1', attributes: {}, description: 'persltr1' },
  }
]

Passing the metadata to a front-end

Going one step further, as a Node.js back-end developer, you now probably want to pass the image regions you have parsed on the back-end to a component on the front-end, like the Image Display Control Web Component, which takes the metadata as a JSON-formatted string.

Let’s adapt our script to produce such a string:

#!/usr/bin/env node
// /tmp/frameright/myscript.mjs

import { promises as fs } from 'fs';

// load the pulled library:
import { Parser } from '@frameright/image-display-control-metadata-parser';

const buffer = await fs.readFile('IPTC-PhotometadataRef-Std2021.1.jpg');
const parser = new Parser(buffer);
const regions = parser.getIdcMetadata('rectangle');

console.log(JSON.stringify(regions, null, 4 /*indentation*/));

Running the script now gives us:

[
    {
        "id": "persltr2",
        "names": [
            "Listener 1"
        ],
        "shape": "rectangle",
        "unit": "relative",
        "x": 0.31,
        "y": 0.18,
        "width": 0.127,
        "height": 0.385
    }
]

Which is exactly the format accepted by the web component:

<img
  id="myimg"
  is="image-display-control"
  src="assets/pics/iptc/IPTC-PhotometadataRef-Std2021.1.jpg"
  data-image-regions='[{
    "id": "persltr2",
    "names": ["Listener 1"],
    "shape": "rectangle",
    "unit": "relative",
    "x": "0.31",
    "y": "0.18",
    "width": "0.127",
    "height": "0.385"
  }]'
/>

Summary

In this tutorial we have learned how to read the Image Region XMP metadata of an image with Node.js by pulling and using a third-party library.