Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1767 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.4956. Min Act: -0.1700

Data Index: 5970575 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.4775. Min Act: -0.1700

Data Index: 2007421 (Open Web Text)

Max Activating Token Index: 583

Text #2

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.3458. Min Act: -0.1700

Data Index: 4337695 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.3061. Min Act: -0.1700

Data Index: 2520831 (Open Web Text)

Max Activating Token Index: 613

Text #4

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.2170. Min Act: -0.1700

Data Index: 4330915 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.2521. Min Act: -0.1700

Data Index: 4017357 (Open Web Text)

Max Activating Token Index: 788

Text #6

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1888. Min Act: -0.1700

Data Index: 2591551 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.2253. Min Act: -0.1700

Data Index: 7546434 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1980. Min Act: -0.1700

Data Index: 7285834 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1948. Min Act: -0.1700

Data Index: 2363283 (Open Web Text)

Max Activating Token Index: 910

Text #10

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1716. Min Act: -0.1700

Data Index: 8612511 (Open Web Text)

Max Activating Token Index: 338

Text #11

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1609. Min Act: -0.1700

Data Index: 1750123 (Open Web Text)

Max Activating Token Index: 169

Text #12

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1281. Min Act: -0.1700

Data Index: 3671237 (Open Web Text)

Max Activating Token Index: 993

Text #13

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1313. Min Act: -0.1700

Data Index: 1763485 (Open Web Text)

Max Activating Token Index: 87

Text #14

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.0898. Min Act: -0.1700

Data Index: 7183915 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1342. Min Act: -0.1700

Data Index: 4676228 (Open Web Text)

Max Activating Token Index: 166

Text #16

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1202. Min Act: -0.1700

Data Index: 5748012 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1212. Min Act: -0.1700

Data Index: 4565476 (Open Web Text)

Max Activating Token Index: 638

Text #18

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1319. Min Act: -0.1700

Data Index: 4414787 (Open Web Text)

Max Activating Token Index: 34

Text #19

Max Range: 2.4956. Min Range: -2.4956

Max Act: 2.1177. Min Act: -0.1700

Data Index: 2649937 (Open Web Text)

Max Activating Token Index: 70