Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 64 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.1231. Min Act: -0.1700

Data Index: 601767 (Open Web Text)

Max Activating Token Index: 962

Text #1

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.1566. Min Act: -0.1700

Data Index: 1672666 (Open Web Text)

Max Activating Token Index: 39

Text #2

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0734. Min Act: -0.1700

Data Index: 3287512 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.1158. Min Act: -0.1700

Data Index: 1906404 (Open Web Text)

Max Activating Token Index: 364

Text #4

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.1060. Min Act: -0.1700

Data Index: 6437075 (Open Web Text)

Max Activating Token Index: 353

Text #5

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0667. Min Act: -0.1700

Data Index: 5296046 (Open Web Text)

Max Activating Token Index: 406

Text #6

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0157. Min Act: -0.1700

Data Index: 7189086 (Open Web Text)

Max Activating Token Index: 511

Text #7

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0358. Min Act: -0.1700

Data Index: 5691947 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0571. Min Act: -0.1700

Data Index: 2833917 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0298. Min Act: -0.1700

Data Index: 5658562 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 3.1566. Min Range: -3.1566

Max Act: 2.9435. Min Act: -0.1700

Data Index: 3947596 (Open Web Text)

Max Activating Token Index: 573

Text #11

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0001. Min Act: -0.1700

Data Index: 5117472 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0143. Min Act: -0.1700

Data Index: 2245719 (Open Web Text)

Max Activating Token Index: 148

Text #13

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0140. Min Act: -0.1700

Data Index: 3551135 (Open Web Text)

Max Activating Token Index: 12

Text #14

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0370. Min Act: -0.1700

Data Index: 3356419 (Open Web Text)

Max Activating Token Index: 474

Text #15

Max Range: 3.1566. Min Range: -3.1566

Max Act: 2.9949. Min Act: -0.1700

Data Index: 3555908 (Open Web Text)

Max Activating Token Index: 35

Text #16

Max Range: 3.1566. Min Range: -3.1566

Max Act: 2.9719. Min Act: -0.1700

Data Index: 5261781 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 3.1566. Min Range: -3.1566

Max Act: 2.9943. Min Act: -0.1700

Data Index: 3859774 (Open Web Text)

Max Activating Token Index: 311

Text #18

Max Range: 3.1566. Min Range: -3.1566

Max Act: 3.0162. Min Act: -0.1700

Data Index: 7287491 (Open Web Text)

Max Activating Token Index: 349

Text #19

Max Range: 3.1566. Min Range: -3.1566

Max Act: 2.9958. Min Act: -0.1700

Data Index: 4758248 (Open Web Text)

Max Activating Token Index: 40