Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 104 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.0487. Min Range: -7.0487

Max Act: 7.0487. Min Act: -0.1700

Data Index: 2198021 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 7.0487. Min Range: -7.0487

Max Act: 7.0182. Min Act: -0.1700

Data Index: 1061445 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 7.0487. Min Range: -7.0487

Max Act: 7.0061. Min Act: -0.1700

Data Index: 6337333 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9456. Min Act: -0.1700

Data Index: 8732589 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9641. Min Act: -0.1700

Data Index: 874501 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9471. Min Act: -0.1700

Data Index: 8620997 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9456. Min Act: -0.1700

Data Index: 7280016 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9956. Min Act: -0.1700

Data Index: 6583969 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9947. Min Act: -0.1700

Data Index: 5607882 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9641. Min Act: -0.1700

Data Index: 731646 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9508. Min Act: -0.1700

Data Index: 5511090 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9701. Min Act: -0.1700

Data Index: 2254156 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9477. Min Act: -0.1700

Data Index: 2399927 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9379. Min Act: -0.1700

Data Index: 2452976 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.8899. Min Act: -0.1700

Data Index: 2503796 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.8899. Min Act: -0.1700

Data Index: 123681 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.8961. Min Act: -0.1700

Data Index: 2919770 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9220. Min Act: -0.1700

Data Index: 3467105 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.9274. Min Act: -0.1700

Data Index: 3947404 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 7.0487. Min Range: -7.0487

Max Act: 6.8961. Min Act: -0.1700

Data Index: 4990289 (Open Web Text)

Max Activating Token Index: 3