Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2829 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.8034. Min Act: -0.0583

Data Index: 6009292 (Open Web Text)

Max Activating Token Index: 657

Text #1

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7438. Min Act: -0.0103

Data Index: 56501 (Open Web Text)

Max Activating Token Index: 962

Text #2

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7333. Min Act: -0.0753

Data Index: 2230237 (Open Web Text)

Max Activating Token Index: 995

Text #3

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7331. Min Act: -0.1614

Data Index: 3417178 (Open Web Text)

Max Activating Token Index: 414

Text #4

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7335. Min Act: -0.1249

Data Index: 5777820 (Open Web Text)

Max Activating Token Index: 982

Text #5

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7358. Min Act: 0.0127

Data Index: 2624780 (Open Web Text)

Max Activating Token Index: 391

Text #6

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7424. Min Act: 0.0011

Data Index: 2100892 (Open Web Text)

Max Activating Token Index: 633

Text #7

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7248. Min Act: -0.1039

Data Index: 7716017 (Open Web Text)

Max Activating Token Index: 711

Text #8

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7113. Min Act: -0.1644

Data Index: 7367364 (Open Web Text)

Max Activating Token Index: 694

Text #9

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.6945. Min Act: -0.0563

Data Index: 5765355 (Open Web Text)

Max Activating Token Index: 1000

Text #10

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7126. Min Act: -0.1101

Data Index: 6479129 (Open Web Text)

Max Activating Token Index: 405

Text #11

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7091. Min Act: -0.0680

Data Index: 498422 (Open Web Text)

Max Activating Token Index: 483

Text #12

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7258. Min Act: -0.0586

Data Index: 2624789 (Open Web Text)

Max Activating Token Index: 1017

Text #13

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7026. Min Act: -0.0851

Data Index: 2318501 (Open Web Text)

Max Activating Token Index: 532

Text #14

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.6974. Min Act: -0.0265

Data Index: 8271777 (Open Web Text)

Max Activating Token Index: 541

Text #15

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7088. Min Act: -0.0530

Data Index: 7079547 (Open Web Text)

Max Activating Token Index: 967

Text #16

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7038. Min Act: -0.1417

Data Index: 4520767 (Open Web Text)

Max Activating Token Index: 148

Text #17

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.6992. Min Act: -0.1437

Data Index: 1304039 (Open Web Text)

Max Activating Token Index: 972

Text #18

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.7033. Min Act: 0.0141

Data Index: 56499 (Open Web Text)

Max Activating Token Index: 219

Text #19

Max Range: 1.8034. Min Range: -1.8034

Max Act: 1.6955. Min Act: -0.1186

Data Index: 2271740 (Open Web Text)

Max Activating Token Index: 420