Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2077 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.1026. Min Act: -0.1700

Data Index: 2362051 (Open Web Text)

Max Activating Token Index: 28

Text #1

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0780. Min Act: -0.1700

Data Index: 1518847 (Open Web Text)

Max Activating Token Index: 310

Text #2

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0411. Min Act: -0.1700

Data Index: 1423575 (Open Web Text)

Max Activating Token Index: 399

Text #3

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0283. Min Act: -0.1700

Data Index: 5649040 (Open Web Text)

Max Activating Token Index: 310

Text #4

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0219. Min Act: -0.1700

Data Index: 6349622 (Open Web Text)

Max Activating Token Index: 193

Text #5

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0180. Min Act: -0.1700

Data Index: 7334655 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0111. Min Act: -0.1700

Data Index: 1661878 (Open Web Text)

Max Activating Token Index: 434

Text #7

Max Range: 5.1026. Min Range: -5.1026

Max Act: 5.0054. Min Act: -0.1700

Data Index: 1282775 (Open Web Text)

Max Activating Token Index: 36

Text #8

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9586. Min Act: -0.1700

Data Index: 3568631 (Open Web Text)

Max Activating Token Index: 11

Text #9

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9622. Min Act: -0.1700

Data Index: 1587867 (Open Web Text)

Max Activating Token Index: 241

Text #10

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9574. Min Act: -0.1700

Data Index: 3893614 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9541. Min Act: -0.1700

Data Index: 1873153 (Open Web Text)

Max Activating Token Index: 374

Text #12

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9238. Min Act: -0.1700

Data Index: 2190481 (Open Web Text)

Max Activating Token Index: 209

Text #13

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9156. Min Act: -0.1700

Data Index: 8018750 (Open Web Text)

Max Activating Token Index: 128

Text #14

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9243. Min Act: -0.1700

Data Index: 4389753 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9251. Min Act: -0.1700

Data Index: 7003319 (Open Web Text)

Max Activating Token Index: 432

Text #16

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9360. Min Act: -0.1700

Data Index: 4432907 (Open Web Text)

Max Activating Token Index: 312

Text #17

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9225. Min Act: -0.1700

Data Index: 54469 (Open Web Text)

Max Activating Token Index: 25

Text #18

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9211. Min Act: -0.1700

Data Index: 3846671 (Open Web Text)

Max Activating Token Index: 223

Text #19

Max Range: 5.1026. Min Range: -5.1026

Max Act: 4.9244. Min Act: -0.1700

Data Index: 8195383 (Open Web Text)

Max Activating Token Index: 7