Loading the File

In order to work on a PNG file the raw data will need to be retrieved. This is done using an XMLHttpRequest with the responseType set to 'arraybuffer'.

const req = new XMLHttpRequest();

req.responseType = 'arraybuffer';  
req.addEventListener('load', (e) => {  
  const arrayBuffer = e.target.response;

  if (!arrayBuffer) {
    throw new Error('No response');
  }

  // this the the byte array to decode
  const byteArray = new Uint8Array(arrayBuffer);
});
req.addEventListener('error', () => {  
  throw new Error('An error has occurred on request');
});

req.open('GET', 'foo.png', true);  
req.send();  

Checking the Signature

A PNG file must start with the following bytes: 137, 80, 78, 71, 13, 10, 26, 10. It is also recommended to check the signature of chunks (explained in the next section) that are constant when possible. This allows us to check the IHDR chunk as it must be the first chunk in a PNG file and the length is always 13.

Given this information, we can check that the PNG file starts with the following bytes: 137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82.

const signature = [137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82];

for (let i = 0; i < signature.length; i++) {  
  if (byteArray[i] !== signature[i]) {
    return false;
  }
}

Reading the Chunk Meta Data

Before diving into how the chunks are read, understanding what a chunk is and the structure of one is required.

A PNG file has multiple chunks starting after the first 8 bytes (these are reserved for the PNG file signature). All chunks are structured in this order:

1. length (4 bytes) - this includes only the length of the data portion of the chunk with a maximum value of 2^31
2. type (4 bytes) - this is the type of chunk (must be treated as binary values)
3. data (X bytes) - data in the chunk, may be 0
4. CRC (4 bytes) - CRC calculated on the preceding bytes in the chunk, excludes length

The IHDR, IDAT, and IEND chunks are required. It is also possible to have multiple chunks of the same type.

Armed with the knowledge of chunks, recall that the check in the previous section there were some bytes appended to the PNG signature. The checked bytes can be seen as the PNG Signature + IHDR Chunk Length + IHDR Chunk Type.

The code below reads all the chunks meta data along with the position of where the data starts to make chunks easier to work with.

// bytesToUint32 converts the given bytes from the "start" to "count" to an unsigned 32 bit integer.
function bytesToUint32(byteArray, start, count) { ... }

const META_SIZE = 4;  
const chunks = [];  
let i = PNG_SIGNATURE.length; // skip the PNG signature  
while (i < byteArray.byteLength) {  
  const dataLength = bytesToUint32(byteArray, i, META_SIZE);

  i += META_SIZE;
  const signature = bytesToString(byteArray, i, META_SIZE);
  const type = chunkSignatureType[signature];

  i += META_SIZE;
  const dataStart = i;

  i += dataLength;
  const crc = bytesToUint32(byteArray, i, META_SIZE);

  i += META_SIZE;

  const meta = {
    type,
    signature,
    crc,
    data: {
      start: dataStart,
      length: dataLength,
    }
  };

  if (type) {
    chunks.push(meta);
  }

  // IEND must be the last chunk
  if (type === 'IEND') {
    break;
  }
}

Parsing IHDR

The IHDR chunk has the following form:

1. Width: 4 bytes
2. Height: 4 bytes
3. Bit depth: 1 byte
4. Color type: 1 byte
5. Compression method: 1 byte
6. Filter method: 1 byte
7. Interlace method: 1 byte

From the start of the data of the chunk, the values can be retrieved by reading the bytes and offsetting by that amount for the next value.

Parsing IDAT

IDAT chunk or chunks contain the image data. It uses an LZ77 derivative and must be decompressed before use. If there are multiple IDAT chunks the data must be concatenated before decompression.

// I used pako for decompressing, 
const compressed = byteArray.slice(start, start + length);  
const decompressed = pako.inflate(compressed);  

Once decompressed, the data can be read though additional work will need to be done. An extra byte is at the start of each scanline (a row of pixels) which tells us what filter was used.

In the case of the sub filter, the following is used:

// filteredBytes represents bytes that have already been unfiltered.
// byteOffset represents the byte to start from in the unfilteredBytes array.
// bytesPerPixel is used to know how many bytes to subtract to get to the previous sample.
function setSubLine(unfilteredBytes, byteOffset, byteArray, start, length, bytesPerPixel) {  
  for (let i = 0; i < length; i++) {
    const offset = byteOffset + i - bytesPerPixel;
    const current = byteArray[start + i];
    const previous = offset >= byteOffset ? unfilteredBytes[offset] : 0;

    unfilteredBytes.push((current + previous) & 0xFF);
  }
}

The unfilteredBytes array now contains the pixel information starting from the upper-left of the image.

More info on filters, documentation can be found here

Summary

Reading a simple PNG in JavaScript is not too bad once the structure is known. I would recommend using a better way to read bytes and converting to them as it seemed to be the part that gave me the most trouble in keeping the code clean.

The finished simple PNG loading code for this post can be found at https://github.com/MWGitHub/basic-loaders/tree/master/src/png

Additional Reading

The PNG specification which can be found here is a great resource on working with the file format.

Unrelated but also interesting is how PNG versioning is done which can be found here

The web application can be found here.
The source code can be found here.

Architecture

Flow

The architecture is somewhat like Flux where data only goes a single direction. The flow of the program is:

• Page loads with blank placeholders
• Fetch request sent which notifies the store
• Store does nothing on send request but passes it to interested listeners for the action
• All listening components update to display the loading animation
• Fetch request finishes and passes the result to the store
• Store sets data and passes it to interested listeners
• Components that are listening asks the store for the data and handles all the required updates

Detail

There are no preset rendering methods. Some components only modify what is already on the DOM and others remove and create on update. Components generally do not modify the DOM or run functions unless a listened event triggers.

There is only a single store since listeners are not store based but action based.

The API utility is the main way to update the store to reduce the complexity of the application though it is still possible to accidentally get stuck in an update loop.

The player on the parent side and the iframe player handles message sending and retrieval from each other. None of the actions directly play songs which allowed for a decoupled way of syncing states between the top tracks and player.

Future Refactors

In the beginning I wanted to create a component that handled binding variables to the DOM with the block.js file but decided to scrap that due to time constraints. block.js now only only handles the passing of information to the change callback when properties are updated. The library does not add much more functionality so I will be refactoring or removing it.

I also created a component elemental.js which creates a pre-made loader. More features will be added to it so that it can take in JSON representing what DOM elements will be created and return the root along with references to the elements when set.

A component base class can be made as some of the code is similar between components. Once the base class is created features can be added through composition instead.

The result:

In this step I compiled gcc 6.1.0 on OS X with the target as x86_64-elf and a suffix so I can have one cross compiler for the kernel and one for user-space. Doing this allows me to build for the target on my machine before the OS can be used for development.

On my first attempt I installed GCC and Binutils without a target. After reading the Why do I need a Cross Compiler? page I decided to build again with a target this time to avoid problems.

To install gcc I followed the steps with some variations from the very useful osdev cross compiler page found here. I built the dependencies (GMP, MPFR, MPC, ISL) separately instead, each with the same prefix path. Now that I have a cross compiler, the next step is to make the kernel.

The first list, "Tasks", is where I create any task that I plan on doing in the future. The second list, "Tomorrow", is a list that queues tasks which move to the third list at the end of the day, which is the "Today" list.

There are only two ways to add tasks in the "Today" list, queuing from the "Tomorrow" list, and creating tasks in the "Today" list as extra tasks. Extra tasks will mainly be used for stat tracking so that I can get a feel of how many tasks I should normally be putting in the queue. Any task in this list will age once per day if not completed. Tasks cannot be deleted until a week has passed, which gives ample time to think about the task before dismissing it. When a task is done I check the task and it moves to the "Complete" list at the end of the day.

The "Complete" list is an immutable list with no interaction. It's just a way to see what you've finished recently.

If you want to try out this workflow I have a web app up for it at:
https://www.bop.xyz/focus/