Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1984 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.2688. Min Act: -0.1700

Data Index: 2828026 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1978. Min Act: -0.1700

Data Index: 3135880 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1428. Min Act: -0.1700

Data Index: 7632000 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1595. Min Act: -0.1700

Data Index: 583117 (Open Web Text)

Max Activating Token Index: 226

Text #4

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1051. Min Act: -0.1700

Data Index: 4084731 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1257. Min Act: -0.1700

Data Index: 8164144 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1205. Min Act: -0.1700

Data Index: 2040735 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1274. Min Act: -0.1700

Data Index: 6911257 (Open Web Text)

Max Activating Token Index: 52

Text #8

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1051. Min Act: -0.1700

Data Index: 2882060 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0780. Min Act: -0.1700

Data Index: 727720 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.1016. Min Act: -0.1700

Data Index: 7499799 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0687. Min Act: -0.1700

Data Index: 8419748 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0598. Min Act: -0.1700

Data Index: 5624563 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0521. Min Act: -0.1700

Data Index: 1828242 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0601. Min Act: -0.1700

Data Index: 518191 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0642. Min Act: -0.1700

Data Index: 3607546 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0415. Min Act: -0.1700

Data Index: 1669208 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0460. Min Act: -0.1700

Data Index: 2413540 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0430. Min Act: -0.1700

Data Index: 7614866 (Open Web Text)

Max Activating Token Index: 22

Text #19

Max Range: 4.2688. Min Range: -4.2688

Max Act: 4.0546. Min Act: -0.1700

Data Index: 7951889 (Open Web Text)

Max Activating Token Index: 4