Sample Api

Download this notebook from GitHub

Preliminaries

Imports

from pathlib import Path

import holoviews as hv
import panel as pn

hv.extension("bokeh")
pn.extension()
# If in google colab, run hack that allows holoviews to work properly
try:
    import google.colab  # noqa

    def _render(self, **kwargs):
        hv.extension("bokeh")
        return hv.Store.render(self)

    hv.core.Dimensioned._repr_mimebundle_ = _render
except ModuleNotFoundError:
    pass

TMP_NOTEBOOK_ROOT = Path("/tmp/bridge-ds/tutorials")

%opts magic unavailable (pyparsing cannot be imported)
%compositor magic unavailable (pyparsing cannot be imported)

ⓘ

Loading a dataset

To create Dataset objects, it’s recommended to utilize a DatasetProvider. In this instance, we’ll employ the Coco2017Detection provider:

from bridge.providers.vision import Coco2017Detection

root_dir = TMP_NOTEBOOK_ROOT / "coco"

provider = Coco2017Detection(root_dir)
ds = provider.build_dataset()
ds

Annotations file /tmp/bridge-ds/tutorials/coco/annotations/instances_train2017.json already exists, skipping download.
loading annotations into memory...
Done (t=14.60s)
creating index...
index created!

Dataset: {'n_samples': 118287, 'n_bbox': 860001, 'n_image': 118287}

Sample API

In BridgeDS, we use two complementing approaches to view datasets. We call them the Sample API and the Table API. This tutorial is about the former.

Sample API can be loosely described as:

A dataset can be viewed as a collection of samples, where each sample is a pythonic object (called Sample) that contains a collection of elements.

In case any of the terms ‘dataset’, ‘sample’ or ‘element’ is foreign to you, we recommend you to first go back to the Key Concepts section.

Let’s demonstrate how to use the Sample API:

Indexing

ds.iget / ds.get are our equivalents of df.iloc / df.loc, used for fetching individual samples from the dataset:

sample = ds.get(34)  # get sample with index 34
print("Sample ID:", sample.id)
sample = ds.iget(1)  # get sample with positional index 1
print("Sample ID:", sample.id)

Sample ID: 34
Sample ID: 25

Properties

The sample object is fairly minimal, exposing only its id, elements, and display_engine properties, with limited methods available. This design reflects its role as a container for elements rather than a data object itself.

Now, let’s shift our focus to the elements:

As a reminder, in BridgeDS, elements can be any type of data—ranging from raw objects like images or text, to annotations like bounding boxes (bboxes), segmentation maps (segmaps), or class labels.

Let’s take a look at the elements contained in our current sample:

print("Sample ID:", sample.id)
print("Total num elements in sample:", len(sample), "\n")
for etype, elist in sample.elements.items():
    print(f"Num elements with etype={etype}:", len(elist))

Sample ID: 25
Total num elements in sample: 3

Num elements with etype=image: 1
Num elements with etype=bbox: 2

We observe one image element and two bbox elements. It’s common to have samples compose of a single element representing raw data (the image) alongside multiple elements representing annotations. To accommodate this frequent use case, we implement COCO samples using a subclass of Sample called SingularSample. This subclass provides a more convenient API, where the main element is accessible via sample.element, and the remaining elements are organized under sample.annotations:

print("The 'sample element' (the image):")
print(f"class: {type(sample).__name__}")
print(f"etype: {sample.element.etype}")
print(f"image shape: {sample.element.data.shape}")
print(f"element_id: {sample.element.id}\n")
print("The annotation elements:")
print(f"n_bboxes: {len(sample.annotations['bbox'])}")
[print(bb_element.data) for bb_element in sample.annotations["bbox"]];

The 'sample element' (the image):
class: SingularSample
etype: image
image shape: (426, 640, 3)
element_id: 25_img

The annotation elements:
n_bboxes: 2
BoundingBox(class_name=25,coords=[385.53  60.03 214.97 297.16]
BoundingBox(class_name=25,coords=[ 53.01 356.49 132.03  55.19]

As you can see, elements are container objects of the actual data - they have the .data property. A sample is just a collection of elements, and is the representation of an individual example from the dataset.

The DisplayEngine

We will elaborate on how the DisplayEngine works in a separate tutorial, but for basic usage it’s worth noting that both the Dataset and the Sample objects expose a .show() method, which takes advantage of the DisplayEngine and produces the following:

sample.show()

# display entire dataset with interactive interface:
ds.show()

As you can see, our class labels are represented as integers rather than strings. This is because the raw COCO dataset stores class labels in this format. If you’d like to learn how to convert these integers into readable strings, or explore how to perform operations across the entire dataset, check out our next tutorial on the Table API.