Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 783 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.5861. Min Act: -0.1700

Data Index: 4674320 (Open Web Text)

Max Activating Token Index: 711

Text #1

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.5565. Min Act: -0.1699

Data Index: 4083844 (Open Web Text)

Max Activating Token Index: 195

Text #2

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.5751. Min Act: -0.1700

Data Index: 3911319 (Open Web Text)

Max Activating Token Index: 545

Text #3

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.5180. Min Act: -0.1700

Data Index: 5140363 (Open Web Text)

Max Activating Token Index: 244

Text #4

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4602. Min Act: -0.1692

Data Index: 2203284 (Open Web Text)

Max Activating Token Index: 964

Text #5

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4930. Min Act: -0.1700

Data Index: 6809151 (Open Web Text)

Max Activating Token Index: 734

Text #6

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4709. Min Act: -0.1700

Data Index: 7456731 (Open Web Text)

Max Activating Token Index: 673

Text #7

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4426. Min Act: -0.1700

Data Index: 226096 (Open Web Text)

Max Activating Token Index: 507

Text #8

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3301. Min Act: -0.1700

Data Index: 8760983 (Open Web Text)

Max Activating Token Index: 369

Text #9

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3793. Min Act: -0.1699

Data Index: 7640945 (Open Web Text)

Max Activating Token Index: 86

Text #10

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4534. Min Act: -0.1700

Data Index: 5498319 (Open Web Text)

Max Activating Token Index: 945

Text #11

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3298. Min Act: -0.1700

Data Index: 5790782 (Open Web Text)

Max Activating Token Index: 811

Text #12

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4607. Min Act: -0.1700

Data Index: 1073495 (Open Web Text)

Max Activating Token Index: 827

Text #13

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4358. Min Act: -0.1700

Data Index: 1347036 (Open Web Text)

Max Activating Token Index: 954

Text #14

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.4157. Min Act: -0.1695

Data Index: 3016133 (Open Web Text)

Max Activating Token Index: 583

Text #15

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3893. Min Act: -0.1700

Data Index: 7692854 (Open Web Text)

Max Activating Token Index: 961

Text #16

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3897. Min Act: -0.1698

Data Index: 6110873 (Open Web Text)

Max Activating Token Index: 527

Text #17

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3881. Min Act: -0.1700

Data Index: 1441788 (Open Web Text)

Max Activating Token Index: 487

Text #18

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3397. Min Act: -0.1700

Data Index: 3793435 (Open Web Text)

Max Activating Token Index: 890

Text #19

Max Range: 5.5861. Min Range: -5.5861

Max Act: 5.3646. Min Act: -0.1700

Data Index: 7065492 (Open Web Text)

Max Activating Token Index: 783