# Intelligence

[View original](https://ittybit.com/docs/intelligence)

***

title: Intelligence
icon: intelligence/magenta
--------------------------

<div className="relative">
  ![Intelligence](https://ittybit-app.ittybitcdn.com/med_49g0uR24sSnA7MXR/search.png)

  <img src="https://ittybit-app.ittybitcdn.com/med_49idRy98vXh05YXC/intelligence-animation.webp" className="absolute w-[33%] -bottom-[11%] -right-6" />
</div>

## Introduction

Ittybit natively supports **Intelligence** for common AI use cases.

Intelligence files are [File Objects](/docs/files) with a `kind` of `speech`, `nsfw`, `description`, `tags`, or `prompt`.

```json
{
  "id": "file_qrstuvwx1234",
  "object": "intelligence",
  "kind": "speech",
  "type": "application/json",
  // ... other props
  "url": "https://you.ittybit.net/example/speech.json",
  "ref": "speech",
}
```

[Media](/docs/media) objects can contain one or more Intelligence files.

```json
{
  "id": "med_abcdefgh1234",
  "object": "media",
  "kind": "video",
  "files": [
    // ... other files
    {
      "id": "file_qrstuvwx1234",
      "object": "intelligence",
      "kind": "speech",
      "type": "application/json",
      "url": "https://you.ittybit.net/example/speech.json",
      "ref": "speech",
    },
    {
      "id": "file_qrstuvwx5678",
      "object": "intelligence",
      "kind": "description",
      "type": "application/json",
      "url": "https://you.ittybit.net/example/description.json",
      "ref": "description",
    }
  ],
  // ... other props
}
```

Intelligence files are formatted as `JSON` with a consistent schema containing rich data about the associated media.

Customers use intelligence files to power features like:

* Transcripts
* NSFW Filtering
* Tags and Descriptions
* Semantic Search
* Recommendations
* Open Graph Previews

***

## Usage

Some customers use ittybit's stored `JSON` files directly. For example, fetching a `speech` file and using the `text` property to display a transcript.

```js
const response = await fetch('https://you.ittybit.net/example/speech.json')
const data = await response.json()
console.log(data.text)
```

Other customers generate intelligence files and then store the values in their database for later use.

```jsx
const item = database.get('articles', { id: 'abcd-1234-efgh-5678' })

return (
  <OpenGraph
    title={item.title}
    description={item.description}
    image={item.urls.thumbnail}
  />
)
```

***

## Kinds

Intelligence files can be one of the following kinds:

* `speech`
* `description`
* `nsfw`
* `prompt`

***

## Speech

Speech files contain a transcript of the spoken content in a video or audio file.

![Speech](https://ittybit-app.ittybitcdn.com/med_49feca86sfddkULE/transcribe.jpg)

```json
{
  "id": "file_qrstuvwx1234",
  "object": "intelligence",
  "kind": "speech",
  "detected": true,
  "speakers": 2,
  "language": "en",
  "text": [
    "Hello, and welcome to UkeTube. I'm Jesse Doe.",
    "And I'm John Doe. Today we're going to be learning Sandstorm by Darude.",
    // ... continues
  ],
  "confidence": 0.72,
  "timeline": [
    {
      "index": 0,
      "start": 12.00,
      "end": 14.50,
      "detected": true,
      "speaker": 0,
      "text": "Hello, and welcome to UkeTube. I'm Jesse Doe.",
      "confidence": 0.89
    },
    {
      "index": 1,
      "start": 14.80,
      "end": 18.28,
      "detected": true,
      "speaker": 1,
      "text": "And I'm John Doe. Today we're going to be learning Sandstorm by Darude.",
      "confidence": 0.91
    },
    // ... continues
  ],
  "created": "2025-01-01T01:23:45Z",
  "updated": "2025-01-01T01:23:45Z",
}
```

See the [Speech Tasks](/docs/speech) section for more information about creating new speech files.

<Card href="/docs/speech" icon={<Icon icon="speech" className="text-brand-magenta" />} title="Speech Tasks" />

***

## Description

Get a text description of the content in an image, video, or audio file.

<div className="grid grid-cols-1 gap-4">
  <img src="https://ittybit-app.ittybitcdn.com/med_49feca91RcW4Kxmi/describe.jpg" className="rounded-lg" />

  <img src="https://ittybit-app.ittybitcdn.com/med_49feca89M4aMDPpE/summarize.jpg" className="rounded-lg" />
</div>

```json
{
  "id": "file_qrstuvwx5678",
  "object": "intelligence",
  "kind": "description",
  "detected": true,
  "language": "en",
  "title": "Designing at Scale: Airbnb's Journey with Design Systems",
  "description": "Karri Saarinen, formerly of Airbnb, presents a compelling case for design systems as a solution to the increasing complexity of product development in the digital age. He argues that design systems are not just about aesthetics, but are crucial tools for organizational efficiency, product consistency, and scalability.",
  "tags": ["Design systems", "Airbnb", "Product development", "Scalability", "UI/UX", "Accessibility", "React Sketchapp", "Organizational change"],
  "confidence": 0.96,
  "created": "2025-01-01T01:23:45Z",
  "updated": "2025-01-01T01:23:45Z",
}
```

See the [Description Tasks](/docs/description) section for more information about creating new description files.

<Card href="/docs/description" icon={<Icon icon="description" className="text-brand-magenta" />} title="Description Tasks" />

***

## NSFW

Detect if a video or image contains explicit content.

![NSFW](https://ittybit-app.ittybitcdn.com/med_49feca912WDIV8w8/nsfw.jpg)

```json
{
  "id": "file_qrstuvwx9012",
  "object": "intelligence",
  "kind": "nsfw",
  "detected": true,
  "nudity": 0.82,
  "sexual": 0.24,
  "violence": 0.64,
  "gore": 0.04,
  "confidence": 0.82,
  "created": "2025-01-01T01:23:45Z",
  "updated": "2025-01-01T01:23:45Z",
}
```

See the [NSFW Tasks](/docs/nsfw) section for more information about creating new nsfw files.

<Card href="/docs/nsfw" icon={<Icon icon="nsfw" className="text-brand-magenta" />} title="NSFW Tasks" />

***

## Prompt

![Prompt](https://ittybit-app.ittybitcdn.com/med_49feca89vkxbP8ma/labels.jpg)

```json
{
  "id": "file_qrstuvwx3456",
  "object": "intelligence",
  "kind": "prompt",
  "detected": true,
  "language": "en",
  "prompt": "Identify the objects in this image. Please respond with JSON and include a bounding box and a confidence score for each item.",
  "response": [
    {
      "label": "White soccer socks",
      "boundingBox": [80, 160, 240, 320],
      "confidence": 0.98
    },
    {
      "label": "Uhlsport Football",
      "boundingBox": [240, 120, 400, 320],
      "confidence": 0.95
    },
    // ... continues
  ],
  "confidence": 0.96,
  "created": "2025-01-01T01:23:45Z",
  "updated": "2025-01-01T01:23:45Z",
}
```

See the [Prompt Tasks](/docs/prompt) section for more information about creating new prompt files.

<Card href="/docs/prompt" icon={<Icon icon="prompt" className="text-brand-magenta" />} title="Prompt Tasks" />

***