Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2430 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4761. Min Act: -0.1700

Data Index: 8661838 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4940. Min Act: -0.1700

Data Index: 2654106 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4466. Min Act: -0.1700

Data Index: 3966292 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4743. Min Act: -0.1700

Data Index: 1715891 (Open Web Text)

Max Activating Token Index: 15

Text #4

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4588. Min Act: -0.1700

Data Index: 8649502 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4467. Min Act: -0.1700

Data Index: 3015277 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4477. Min Act: -0.1700

Data Index: 4599019 (Open Web Text)

Max Activating Token Index: 16

Text #7

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4273. Min Act: -0.1700

Data Index: 2967963 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4342. Min Act: -0.1700

Data Index: 557496 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4292. Min Act: -0.1700

Data Index: 1581788 (Open Web Text)

Max Activating Token Index: 13

Text #10

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4194. Min Act: -0.1700

Data Index: 445392 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4200. Min Act: -0.1700

Data Index: 2691271 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4338. Min Act: -0.1700

Data Index: 5103655 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4167. Min Act: -0.1700

Data Index: 6809696 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.3924. Min Act: -0.1700

Data Index: 2768936 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.3747. Min Act: -0.1700

Data Index: 5198147 (Open Web Text)

Max Activating Token Index: 17

Text #16

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.3891. Min Act: -0.1700

Data Index: 2656192 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.3868. Min Act: -0.1700

Data Index: 2865498 (Open Web Text)

Max Activating Token Index: 8

Text #18

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.3982. Min Act: -0.1700

Data Index: 2236108 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 3.4940. Min Range: -3.4940

Max Act: 3.4051. Min Act: -0.1700

Data Index: 6339126 (Open Web Text)

Max Activating Token Index: 26