{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Preliminaries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import holoviews as hv\n",
    "import panel as pn\n",
    "\n",
    "hv.extension(\"bokeh\")\n",
    "pn.extension()\n",
    "# If in google colab, run hack that allows holoviews to work properly\n",
    "try:\n",
    "    import google.colab  # noqa\n",
    "\n",
    "    def _render(self, **kwargs):\n",
    "        hv.extension(\"bokeh\")\n",
    "        return hv.Store.render(self)\n",
    "\n",
    "    hv.core.Dimensioned._repr_mimebundle_ = _render\n",
    "except ModuleNotFoundError:\n",
    "    pass\n",
    "\n",
    "TMP_NOTEBOOK_ROOT = Path(\"/tmp/bridge-ds/tutorials\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "## Loading a dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {},
   "source": [
    "To create BridgeDS Dataset objects, it's recommended to utilize a **DatasetProvider**. In this instance, we'll employ the Coco2017Detection provider:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bridge.providers.vision import Coco2017Detection\n",
    "\n",
    "root_dir = TMP_NOTEBOOK_ROOT / \"coco\"\n",
    "\n",
    "provider = Coco2017Detection(root_dir, split=\"train\", img_source=\"stream\")\n",
    "ds = provider.build_dataset()\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "# TableAPI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "In BridgeDS, we use two complementing approaches to view datasets. We call them the **Sample API** and the **Table API**. This tutorial is about the latter.\n",
    "\n",
    "The Table API can be described as:\n",
    "\n",
    "    A dataset can be viewed as a table where every row represents a single element. Elements have a unique element_id but share their sample_id with other elements from the same Sample. The element_id and sample_id columns serve as the table's multi-index."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "## Tables\n",
    "Like in the previous tutorial, we semantically split the elements into two groups: `ds.samples` containing images and `ds.annotations` containing bboxes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.samples.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.annotations.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "## Methods\n",
    "The Table API is designed to expose callables that accept Pandas DataFrames as arguments, due to their simple and familiar API. The following sections showcase methods that allow users to perform different actions on Datasets, and these methods accept tuples of DataFrames (samples, annotations).\n",
    "\n",
    "### Filter\n",
    "Using tables allows us to easily filter out images or bboxes using familiar Pandas syntax. Note that when filtering samples, BridgeDS automatically filters out corresponding annotations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Original dataset:\")\n",
    "print(ds, \"\\n\")\n",
    "print(\"Filter out images (and corresponding bboxes) where the license < 3:\")\n",
    "print(ds.select_samples(lambda samples, anns: samples.license < 3), \"\\n\")\n",
    "print(\"Filter all bboxes with iscrowd==0. This leaves us with some empty images:\")\n",
    "print(ds.select_annotations(lambda samples, anns: anns.iscrowd != 0), \"\\n\")\n",
    "print(\"We can pipe both selectors to filter out the bboxes, and subsequently filter out empty images:\")\n",
    "print(\n",
    "    ds.select_annotations(lambda samples, anns: anns.iscrowd != 0).select_samples(\n",
    "        lambda samples, anns: samples.index.get_level_values(\"sample_id\").isin(anns.index.get_level_values(\"sample_id\"))\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "### Assign\n",
    "We can assign new columns to either `ds.samples` or `ds.annotations` using familiar syntax. Let's assign the value `n_bboxes` to every sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = ds.assign_samples(\n",
    "    n_bboxes=lambda samples, anns: anns.groupby(\"sample_id\")\n",
    "    .size()\n",
    "    .reindex(samples.index.get_level_values(\"sample_id\"), fill_value=0)\n",
    "    .values\n",
    ")\n",
    "ds.samples.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15",
   "metadata": {},
   "source": [
    "### Sorting\n",
    "We can sort the tables using familiar Pandas syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "sorted_ds = ds.sort_samples(\"n_bboxes\", ascending=False)\n",
    "sorted_ds.samples.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17",
   "metadata": {},
   "source": [
    "Note that if we sort the samples table, we can change the positional index used by the Sample API (ds.iget) which dictates the order of the samples below. The next cell will show the dataset in order from most bboxes per image to least:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18",
   "metadata": {},
   "outputs": [],
   "source": [
    "sorted_ds.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}