Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2337 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.8149. Min Act: -0.1700

Data Index: 1557666 (Open Web Text)

Max Activating Token Index: 712

Text #1

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6471. Min Act: -0.1700

Data Index: 8742429 (Open Web Text)

Max Activating Token Index: 894

Text #2

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6404. Min Act: -0.1700

Data Index: 2377286 (Open Web Text)

Max Activating Token Index: 861

Text #3

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6254. Min Act: -0.1700

Data Index: 7120642 (Open Web Text)

Max Activating Token Index: 168

Text #4

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6055. Min Act: -0.1700

Data Index: 8048135 (Open Web Text)

Max Activating Token Index: 858

Text #5

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6228. Min Act: -0.1700

Data Index: 4012885 (Open Web Text)

Max Activating Token Index: 733

Text #6

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6008. Min Act: -0.1700

Data Index: 2163941 (Open Web Text)

Max Activating Token Index: 839

Text #7

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5752. Min Act: -0.1700

Data Index: 59942 (Open Web Text)

Max Activating Token Index: 765

Text #8

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5877. Min Act: -0.1700

Data Index: 6620771 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5886. Min Act: -0.1700

Data Index: 4125327 (Open Web Text)

Max Activating Token Index: 875

Text #10

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5865. Min Act: -0.1700

Data Index: 561951 (Open Web Text)

Max Activating Token Index: 669

Text #11

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.6061. Min Act: -0.1700

Data Index: 2940241 (Open Web Text)

Max Activating Token Index: 12

Text #12

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5752. Min Act: -0.1700

Data Index: 3756374 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5810. Min Act: -0.1700

Data Index: 1878954 (Open Web Text)

Max Activating Token Index: 937

Text #14

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5733. Min Act: -0.1700

Data Index: 6092396 (Open Web Text)

Max Activating Token Index: 465

Text #15

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5553. Min Act: -0.1700

Data Index: 1504857 (Open Web Text)

Max Activating Token Index: 762

Text #16

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5670. Min Act: -0.1700

Data Index: 408026 (Open Web Text)

Max Activating Token Index: 96

Text #17

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5843. Min Act: -0.1700

Data Index: 1881346 (Open Web Text)

Max Activating Token Index: 980

Text #18

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5598. Min Act: -0.1700

Data Index: 3246542 (Open Web Text)

Max Activating Token Index: 942

Text #19

Max Range: 3.8149. Min Range: -3.8149

Max Act: 3.5456. Min Act: -0.1700

Data Index: 4209111 (Open Web Text)

Max Activating Token Index: 954