Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 388 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.3859. Min Act: -0.1700

Data Index: 4872739 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.2649. Min Act: -0.1700

Data Index: 1907106 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.2355. Min Act: -0.1700

Data Index: 8156527 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.2311. Min Act: -0.1700

Data Index: 8103920 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.2237. Min Act: -0.1700

Data Index: 5950265 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1941. Min Act: -0.1700

Data Index: 4116128 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1578. Min Act: -0.1700

Data Index: 7511371 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1613. Min Act: -0.1700

Data Index: 1597083 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1727. Min Act: -0.1700

Data Index: 7779153 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1425. Min Act: -0.1700

Data Index: 6373714 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1250. Min Act: -0.1700

Data Index: 6037685 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1271. Min Act: -0.1700

Data Index: 7885162 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1252. Min Act: -0.1700

Data Index: 7246454 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1530. Min Act: -0.1700

Data Index: 2804958 (Open Web Text)

Max Activating Token Index: 702

Text #14

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.0984. Min Act: -0.1700

Data Index: 5139997 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1015. Min Act: -0.1700

Data Index: 8275094 (Open Web Text)

Max Activating Token Index: 15

Text #16

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.1151. Min Act: -0.1700

Data Index: 8461146 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.0734. Min Act: -0.1700

Data Index: 1561601 (Open Web Text)

Max Activating Token Index: 12

Text #18

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.0544. Min Act: -0.1700

Data Index: 5810172 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 4.3859. Min Range: -4.3859

Max Act: 4.0726. Min Act: -0.1700

Data Index: 7759887 (Open Web Text)

Max Activating Token Index: 3