Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2108 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.5208. Min Act: -0.1700

Data Index: 2629621 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.5190. Min Act: -0.1700

Data Index: 490905 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.4557. Min Act: -0.1700

Data Index: 3551544 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.4557. Min Act: -0.1700

Data Index: 7204780 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3641. Min Act: -0.1700

Data Index: 5093491 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3762. Min Act: -0.1700

Data Index: 6175874 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3695. Min Act: -0.1700

Data Index: 1091333 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3560. Min Act: -0.1700

Data Index: 7427368 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3717. Min Act: -0.1700

Data Index: 152797 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3385. Min Act: -0.1700

Data Index: 522788 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3421. Min Act: -0.1700

Data Index: 8561590 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2956. Min Act: -0.1700

Data Index: 4404242 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2869. Min Act: -0.1700

Data Index: 5842499 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2869. Min Act: -0.1700

Data Index: 700694 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2869. Min Act: -0.1700

Data Index: 6826981 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3018. Min Act: -0.1700

Data Index: 6557756 (Open Web Text)

Max Activating Token Index: 11

Text #16

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2941. Min Act: -0.1700

Data Index: 6354660 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.3115. Min Act: -0.1700

Data Index: 6287041 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2628. Min Act: -0.1700

Data Index: 738027 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.5208. Min Range: -5.5208

Max Act: 5.2753. Min Act: -0.1700

Data Index: 2149262 (Open Web Text)

Max Activating Token Index: 7