Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1400 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.3644. Min Act: -0.1700

Data Index: 1898890 (Open Web Text)

Max Activating Token Index: 68

Text #1

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.3560. Min Act: -0.1700

Data Index: 6288323 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.3181. Min Act: -0.1700

Data Index: 4825229 (Open Web Text)

Max Activating Token Index: 358

Text #3

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2958. Min Act: -0.1700

Data Index: 5758834 (Open Web Text)

Max Activating Token Index: 174

Text #4

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2829. Min Act: -0.1700

Data Index: 5445277 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2818. Min Act: -0.1700

Data Index: 1072497 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2728. Min Act: -0.1700

Data Index: 3449773 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2705. Min Act: -0.1700

Data Index: 2443068 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2663. Min Act: -0.1700

Data Index: 5458053 (Open Web Text)

Max Activating Token Index: 91

Text #9

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2723. Min Act: -0.1700

Data Index: 7264834 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2549. Min Act: -0.1700

Data Index: 1510697 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2392. Min Act: -0.1700

Data Index: 8783209 (Open Web Text)

Max Activating Token Index: 395

Text #12

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2367. Min Act: -0.1700

Data Index: 8023159 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2194. Min Act: -0.1700

Data Index: 1847951 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2106. Min Act: -0.1700

Data Index: 4257190 (Open Web Text)

Max Activating Token Index: 23

Text #15

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2140. Min Act: -0.1700

Data Index: 960338 (Open Web Text)

Max Activating Token Index: 2

Text #16

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.1880. Min Act: -0.1700

Data Index: 318137 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.2136. Min Act: -0.1700

Data Index: 5167330 (Open Web Text)

Max Activating Token Index: 448

Text #18

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.1856. Min Act: -0.1700

Data Index: 5102127 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 2.3644. Min Range: -2.3644

Max Act: 2.1856. Min Act: -0.1700

Data Index: 4852325 (Open Web Text)

Max Activating Token Index: 5