Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2448 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.4484. Min Act: -0.1700

Data Index: 1723886 (Open Web Text)

Max Activating Token Index: 508

Text #1

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.4222. Min Act: -0.1700

Data Index: 8618390 (Open Web Text)

Max Activating Token Index: 201

Text #2

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.4079. Min Act: -0.1700

Data Index: 6865615 (Open Web Text)

Max Activating Token Index: 575

Text #3

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3630. Min Act: -0.1700

Data Index: 8248433 (Open Web Text)

Max Activating Token Index: 519

Text #4

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3558. Min Act: -0.1700

Data Index: 5192902 (Open Web Text)

Max Activating Token Index: 396

Text #5

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3529. Min Act: -0.1700

Data Index: 4812873 (Open Web Text)

Max Activating Token Index: 623

Text #6

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3396. Min Act: -0.1700

Data Index: 3895911 (Open Web Text)

Max Activating Token Index: 481

Text #7

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3264. Min Act: -0.1700

Data Index: 6313246 (Open Web Text)

Max Activating Token Index: 802

Text #8

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3451. Min Act: -0.1700

Data Index: 1503661 (Open Web Text)

Max Activating Token Index: 659

Text #9

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3287. Min Act: -0.1700

Data Index: 6840560 (Open Web Text)

Max Activating Token Index: 481

Text #10

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3151. Min Act: -0.1700

Data Index: 250330 (Open Web Text)

Max Activating Token Index: 910

Text #11

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3074. Min Act: -0.1700

Data Index: 7259481 (Open Web Text)

Max Activating Token Index: 534

Text #12

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3108. Min Act: -0.1700

Data Index: 3880434 (Open Web Text)

Max Activating Token Index: 410

Text #13

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3151. Min Act: -0.1700

Data Index: 3142420 (Open Web Text)

Max Activating Token Index: 598

Text #14

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3064. Min Act: -0.1700

Data Index: 2481882 (Open Web Text)

Max Activating Token Index: 13

Text #15

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3016. Min Act: -0.1700

Data Index: 7778207 (Open Web Text)

Max Activating Token Index: 398

Text #16

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3297. Min Act: -0.1700

Data Index: 2599654 (Open Web Text)

Max Activating Token Index: 426

Text #17

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.2974. Min Act: -0.1700

Data Index: 2652833 (Open Web Text)

Max Activating Token Index: 315

Text #18

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3086. Min Act: -0.1700

Data Index: 2001926 (Open Web Text)

Max Activating Token Index: 351

Text #19

Max Range: 3.4484. Min Range: -3.4484

Max Act: 3.3002. Min Act: -0.1700

Data Index: 128611 (Open Web Text)

Max Activating Token Index: 400