Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 498 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7809. Min Act: -0.1700

Data Index: 6619977 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7996. Min Act: -0.1700

Data Index: 3257542 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.8018. Min Act: -0.1700

Data Index: 613008 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7649. Min Act: -0.1700

Data Index: 3945317 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7484. Min Act: -0.1700

Data Index: 2213905 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7646. Min Act: -0.1700

Data Index: 7829673 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7704. Min Act: -0.1700

Data Index: 7639433 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7469. Min Act: -0.1700

Data Index: 6262128 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7537. Min Act: -0.1700

Data Index: 3219269 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7510. Min Act: -0.1700

Data Index: 6167171 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7594. Min Act: -0.1700

Data Index: 5447164 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7570. Min Act: -0.1700

Data Index: 2977502 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7750. Min Act: -0.1700

Data Index: 5238359 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7650. Min Act: -0.1700

Data Index: 2747521 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7791. Min Act: -0.1700

Data Index: 4302526 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7696. Min Act: -0.1700

Data Index: 4199163 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7703. Min Act: -0.1700

Data Index: 1478263 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7673. Min Act: -0.1700

Data Index: 2491574 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7297. Min Act: -0.1700

Data Index: 489183 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 3.8018. Min Range: -3.8018

Max Act: 3.7201. Min Act: -0.1700

Data Index: 2295687 (Open Web Text)

Max Activating Token Index: 5