Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1401 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.5738. Min Act: -0.1700

Data Index: 2146774 (Open Web Text)

Max Activating Token Index: 510

Text #1

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.5372. Min Act: -0.1700

Data Index: 447521 (Open Web Text)

Max Activating Token Index: 453

Text #2

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.5565. Min Act: -0.1700

Data Index: 6508248 (Open Web Text)

Max Activating Token Index: 1003

Text #3

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.5321. Min Act: -0.1700

Data Index: 860215 (Open Web Text)

Max Activating Token Index: 535

Text #4

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4940. Min Act: -0.1700

Data Index: 3916205 (Open Web Text)

Max Activating Token Index: 272

Text #5

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.5220. Min Act: -0.1700

Data Index: 5989425 (Open Web Text)

Max Activating Token Index: 49

Text #6

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4808. Min Act: -0.1700

Data Index: 379881 (Open Web Text)

Max Activating Token Index: 548

Text #7

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4193. Min Act: -0.1700

Data Index: 3355631 (Open Web Text)

Max Activating Token Index: 210

Text #8

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4428. Min Act: -0.1700

Data Index: 602374 (Open Web Text)

Max Activating Token Index: 124

Text #9

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4867. Min Act: -0.1700

Data Index: 3099973 (Open Web Text)

Max Activating Token Index: 142

Text #10

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4570. Min Act: -0.1700

Data Index: 2246088 (Open Web Text)

Max Activating Token Index: 920

Text #11

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4121. Min Act: -0.1700

Data Index: 6779542 (Open Web Text)

Max Activating Token Index: 545

Text #12

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4530. Min Act: -0.1700

Data Index: 4311148 (Open Web Text)

Max Activating Token Index: 922

Text #13

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4257. Min Act: -0.1700

Data Index: 1309048 (Open Web Text)

Max Activating Token Index: 798

Text #14

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4171. Min Act: -0.1700

Data Index: 4152230 (Open Web Text)

Max Activating Token Index: 804

Text #15

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.4103. Min Act: -0.1700

Data Index: 5056808 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.3888. Min Act: -0.1700

Data Index: 4591265 (Open Web Text)

Max Activating Token Index: 929

Text #17

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.3848. Min Act: -0.1700

Data Index: 1607993 (Open Web Text)

Max Activating Token Index: 888

Text #18

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.3736. Min Act: -0.1700

Data Index: 1013176 (Open Web Text)

Max Activating Token Index: 1003

Text #19

Max Range: 5.5738. Min Range: -5.5738

Max Act: 5.3665. Min Act: -0.1700

Data Index: 6709825 (Open Web Text)

Max Activating Token Index: 320