Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 705 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4806. Min Act: -0.1700

Data Index: 6093402 (Open Web Text)

Max Activating Token Index: 47

Text #1

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4519. Min Act: -0.1693

Data Index: 8790166 (Open Web Text)

Max Activating Token Index: 122

Text #2

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4534. Min Act: -0.1700

Data Index: 3022699 (Open Web Text)

Max Activating Token Index: 770

Text #3

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4590. Min Act: -0.1688

Data Index: 7672865 (Open Web Text)

Max Activating Token Index: 800

Text #4

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4314. Min Act: -0.1700

Data Index: 6531461 (Open Web Text)

Max Activating Token Index: 788

Text #5

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4348. Min Act: -0.1672

Data Index: 3208461 (Open Web Text)

Max Activating Token Index: 538

Text #6

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4261. Min Act: -0.1700

Data Index: 3880700 (Open Web Text)

Max Activating Token Index: 793

Text #7

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4151. Min Act: -0.1700

Data Index: 3651179 (Open Web Text)

Max Activating Token Index: 709

Text #8

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4194. Min Act: -0.1700

Data Index: 1952711 (Open Web Text)

Max Activating Token Index: 563

Text #9

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4086. Min Act: -0.1693

Data Index: 1476831 (Open Web Text)

Max Activating Token Index: 645

Text #10

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4008. Min Act: -0.1697

Data Index: 683142 (Open Web Text)

Max Activating Token Index: 864

Text #11

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4076. Min Act: -0.1700

Data Index: 1297351 (Open Web Text)

Max Activating Token Index: 360

Text #12

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4002. Min Act: -0.1697

Data Index: 301059 (Open Web Text)

Max Activating Token Index: 132

Text #13

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4067. Min Act: -0.1700

Data Index: 6938532 (Open Web Text)

Max Activating Token Index: 994

Text #14

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4027. Min Act: -0.1699

Data Index: 6709072 (Open Web Text)

Max Activating Token Index: 308

Text #15

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.3962. Min Act: -0.1693

Data Index: 6164014 (Open Web Text)

Max Activating Token Index: 723

Text #16

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4015. Min Act: -0.1698

Data Index: 7855369 (Open Web Text)

Max Activating Token Index: 1022

Text #17

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.3998. Min Act: -0.1700

Data Index: 6655467 (Open Web Text)

Max Activating Token Index: 703

Text #18

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.4034. Min Act: -0.1689

Data Index: 1296271 (Open Web Text)

Max Activating Token Index: 437

Text #19

Max Range: 1.4806. Min Range: -1.4806

Max Act: 1.3982. Min Act: -0.1699

Data Index: 8243413 (Open Web Text)

Max Activating Token Index: 692