Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2918 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.5070. Min Act: -0.1700

Data Index: 3784935 (Open Web Text)

Max Activating Token Index: 375

Text #1

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.4142. Min Act: -0.1700

Data Index: 5985153 (Open Web Text)

Max Activating Token Index: 432

Text #2

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3653. Min Act: -0.1700

Data Index: 7267108 (Open Web Text)

Max Activating Token Index: 38

Text #3

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3525. Min Act: -0.1700

Data Index: 8207035 (Open Web Text)

Max Activating Token Index: 248

Text #4

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3467. Min Act: -0.1700

Data Index: 6917247 (Open Web Text)

Max Activating Token Index: 610

Text #5

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3666. Min Act: -0.1700

Data Index: 5004897 (Open Web Text)

Max Activating Token Index: 600

Text #6

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3729. Min Act: -0.1700

Data Index: 6282568 (Open Web Text)

Max Activating Token Index: 56

Text #7

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3053. Min Act: -0.1700

Data Index: 6226871 (Open Web Text)

Max Activating Token Index: 574

Text #8

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3503. Min Act: -0.1700

Data Index: 773936 (Open Web Text)

Max Activating Token Index: 245

Text #9

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3527. Min Act: -0.1700

Data Index: 3823219 (Open Web Text)

Max Activating Token Index: 248

Text #10

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3451. Min Act: -0.1700

Data Index: 6138860 (Open Web Text)

Max Activating Token Index: 744

Text #11

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3616. Min Act: -0.1700

Data Index: 1553037 (Open Web Text)

Max Activating Token Index: 472

Text #12

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3398. Min Act: -0.1700

Data Index: 5384536 (Open Web Text)

Max Activating Token Index: 301

Text #13

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3273. Min Act: -0.1700

Data Index: 7919691 (Open Web Text)

Max Activating Token Index: 369

Text #14

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3326. Min Act: -0.1700

Data Index: 6103650 (Open Web Text)

Max Activating Token Index: 154

Text #15

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3499. Min Act: -0.1700

Data Index: 8378062 (Open Web Text)

Max Activating Token Index: 275

Text #16

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3141. Min Act: -0.1700

Data Index: 8234225 (Open Web Text)

Max Activating Token Index: 382

Text #17

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.3266. Min Act: -0.1700

Data Index: 2353583 (Open Web Text)

Max Activating Token Index: 507

Text #18

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.2978. Min Act: -0.1700

Data Index: 4549942 (Open Web Text)

Max Activating Token Index: 167

Text #19

Max Range: 4.5070. Min Range: -4.5070

Max Act: 4.2698. Min Act: -0.1700

Data Index: 3421719 (Open Web Text)

Max Activating Token Index: 885