Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 230 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.6816. Min Act: -0.1700

Data Index: 1436841 (Open Web Text)

Max Activating Token Index: 731

Text #1

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.6240. Min Act: -0.1700

Data Index: 8066945 (Open Web Text)

Max Activating Token Index: 581

Text #2

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.6303. Min Act: -0.1700

Data Index: 2039619 (Open Web Text)

Max Activating Token Index: 693

Text #3

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4644. Min Act: -0.1700

Data Index: 2170462 (Open Web Text)

Max Activating Token Index: 153

Text #4

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4864. Min Act: -0.1700

Data Index: 103206 (Open Web Text)

Max Activating Token Index: 303

Text #5

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.5112. Min Act: -0.1700

Data Index: 4286693 (Open Web Text)

Max Activating Token Index: 952

Text #6

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4764. Min Act: -0.1700

Data Index: 4683187 (Open Web Text)

Max Activating Token Index: 660

Text #7

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4536. Min Act: -0.1700

Data Index: 5936363 (Open Web Text)

Max Activating Token Index: 875

Text #8

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4828. Min Act: -0.1700

Data Index: 1405112 (Open Web Text)

Max Activating Token Index: 696

Text #9

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4273. Min Act: -0.1700

Data Index: 5736966 (Open Web Text)

Max Activating Token Index: 941

Text #10

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4431. Min Act: -0.1700

Data Index: 5389526 (Open Web Text)

Max Activating Token Index: 791

Text #11

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3866. Min Act: -0.1700

Data Index: 615758 (Open Web Text)

Max Activating Token Index: 437

Text #12

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3893. Min Act: -0.1700

Data Index: 2477895 (Open Web Text)

Max Activating Token Index: 658

Text #13

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3569. Min Act: -0.1700

Data Index: 6782105 (Open Web Text)

Max Activating Token Index: 435

Text #14

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4052. Min Act: -0.1700

Data Index: 5042518 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3923. Min Act: -0.1700

Data Index: 1868978 (Open Web Text)

Max Activating Token Index: 77

Text #16

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3749. Min Act: -0.1700

Data Index: 5492982 (Open Web Text)

Max Activating Token Index: 395

Text #17

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.3714. Min Act: -0.1700

Data Index: 3802403 (Open Web Text)

Max Activating Token Index: 537

Text #18

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4029. Min Act: -0.1700

Data Index: 6428547 (Open Web Text)

Max Activating Token Index: 500

Text #19

Max Range: 7.6816. Min Range: -7.6816

Max Act: 7.4016. Min Act: -0.1700

Data Index: 3523133 (Open Web Text)

Max Activating Token Index: 187