Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1214 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.6581. Min Act: -0.1700

Data Index: 1873790 (Open Web Text)

Max Activating Token Index: 862

Text #1

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.5642. Min Act: -0.1700

Data Index: 4194026 (Open Web Text)

Max Activating Token Index: 319

Text #2

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.5493. Min Act: -0.1700

Data Index: 5571218 (Open Web Text)

Max Activating Token Index: 838

Text #3

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.5698. Min Act: -0.1700

Data Index: 849090 (Open Web Text)

Max Activating Token Index: 645

Text #4

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4956. Min Act: -0.1700

Data Index: 3768083 (Open Web Text)

Max Activating Token Index: 507

Text #5

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.5041. Min Act: -0.1700

Data Index: 1385287 (Open Web Text)

Max Activating Token Index: 281

Text #6

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.5039. Min Act: -0.1700

Data Index: 8238993 (Open Web Text)

Max Activating Token Index: 843

Text #7

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4350. Min Act: -0.1700

Data Index: 1672835 (Open Web Text)

Max Activating Token Index: 850

Text #8

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4587. Min Act: -0.1700

Data Index: 2624535 (Open Web Text)

Max Activating Token Index: 543

Text #9

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4789. Min Act: -0.1700

Data Index: 8216125 (Open Web Text)

Max Activating Token Index: 512

Text #10

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4409. Min Act: -0.1700

Data Index: 1738518 (Open Web Text)

Max Activating Token Index: 214

Text #11

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4215. Min Act: -0.1700

Data Index: 3125900 (Open Web Text)

Max Activating Token Index: 187

Text #12

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4217. Min Act: -0.1700

Data Index: 6998569 (Open Web Text)

Max Activating Token Index: 272

Text #13

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4142. Min Act: -0.1700

Data Index: 6680513 (Open Web Text)

Max Activating Token Index: 437

Text #14

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4020. Min Act: -0.1700

Data Index: 7133996 (Open Web Text)

Max Activating Token Index: 734

Text #15

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.3920. Min Act: -0.1700

Data Index: 4637828 (Open Web Text)

Max Activating Token Index: 1020

Text #16

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4216. Min Act: -0.1700

Data Index: 7432714 (Open Web Text)

Max Activating Token Index: 575

Text #17

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4047. Min Act: -0.1700

Data Index: 4812142 (Open Web Text)

Max Activating Token Index: 870

Text #18

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4117. Min Act: -0.1700

Data Index: 7642135 (Open Web Text)

Max Activating Token Index: 213

Text #19

Max Range: 4.6581. Min Range: -4.6581

Max Act: 4.4031. Min Act: -0.1700

Data Index: 7985185 (Open Web Text)

Max Activating Token Index: 582