Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2724 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.5251. Min Act: -0.1700

Data Index: 2939714 (Open Web Text)

Max Activating Token Index: 471

Text #1

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.5459. Min Act: -0.1700

Data Index: 1122643 (Open Web Text)

Max Activating Token Index: 1005

Text #2

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.5405. Min Act: -0.1700

Data Index: 7999089 (Open Web Text)

Max Activating Token Index: 503

Text #3

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.5091. Min Act: -0.1700

Data Index: 5074225 (Open Web Text)

Max Activating Token Index: 368

Text #4

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4858. Min Act: -0.1700

Data Index: 2214954 (Open Web Text)

Max Activating Token Index: 605

Text #5

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4875. Min Act: -0.1700

Data Index: 1330021 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4751. Min Act: -0.1700

Data Index: 195405 (Open Web Text)

Max Activating Token Index: 367

Text #7

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4806. Min Act: -0.1700

Data Index: 526158 (Open Web Text)

Max Activating Token Index: 731

Text #8

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4911. Min Act: -0.1700

Data Index: 5091117 (Open Web Text)

Max Activating Token Index: 36

Text #9

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4835. Min Act: -0.1700

Data Index: 8249578 (Open Web Text)

Max Activating Token Index: 608

Text #10

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4459. Min Act: -0.1700

Data Index: 4747384 (Open Web Text)

Max Activating Token Index: 435

Text #11

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4542. Min Act: -0.1700

Data Index: 8697743 (Open Web Text)

Max Activating Token Index: 787

Text #12

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4504. Min Act: -0.1700

Data Index: 556142 (Open Web Text)

Max Activating Token Index: 981

Text #13

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4745. Min Act: -0.1700

Data Index: 147866 (Open Web Text)

Max Activating Token Index: 576

Text #14

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4290. Min Act: -0.1700

Data Index: 8736295 (Open Web Text)

Max Activating Token Index: 91

Text #15

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4583. Min Act: -0.1700

Data Index: 5212304 (Open Web Text)

Max Activating Token Index: 727

Text #16

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4227. Min Act: -0.1700

Data Index: 2491421 (Open Web Text)

Max Activating Token Index: 400

Text #17

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4530. Min Act: -0.1700

Data Index: 5140230 (Open Web Text)

Max Activating Token Index: 51

Text #18

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4292. Min Act: -0.1700

Data Index: 1616118 (Open Web Text)

Max Activating Token Index: 280

Text #19

Max Range: 3.5459. Min Range: -3.5459

Max Act: 3.4125. Min Act: -0.1700

Data Index: 4101335 (Open Web Text)

Max Activating Token Index: 124