Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2600 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.4912. Min Act: -0.1700

Data Index: 1599014 (Open Web Text)

Max Activating Token Index: 594

Text #1

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.3182. Min Act: -0.1700

Data Index: 8086838 (Open Web Text)

Max Activating Token Index: 929

Text #2

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.3213. Min Act: -0.1700

Data Index: 3512358 (Open Web Text)

Max Activating Token Index: 397

Text #3

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.3354. Min Act: -0.1700

Data Index: 2721321 (Open Web Text)

Max Activating Token Index: 651

Text #4

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.2652. Min Act: -0.1700

Data Index: 2964831 (Open Web Text)

Max Activating Token Index: 227

Text #5

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.3262. Min Act: -0.1700

Data Index: 2039821 (Open Web Text)

Max Activating Token Index: 331

Text #6

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.2431. Min Act: -0.1700

Data Index: 6709502 (Open Web Text)

Max Activating Token Index: 650

Text #7

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1964. Min Act: -0.1700

Data Index: 1959976 (Open Web Text)

Max Activating Token Index: 365

Text #8

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1964. Min Act: -0.1700

Data Index: 100374 (Open Web Text)

Max Activating Token Index: 889

Text #9

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1523. Min Act: -0.1700

Data Index: 6690441 (Open Web Text)

Max Activating Token Index: 502

Text #10

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1532. Min Act: -0.1700

Data Index: 8232062 (Open Web Text)

Max Activating Token Index: 507

Text #11

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1989. Min Act: -0.1700

Data Index: 2954308 (Open Web Text)

Max Activating Token Index: 677

Text #12

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1377. Min Act: -0.1700

Data Index: 4186091 (Open Web Text)

Max Activating Token Index: 501

Text #13

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.1034. Min Act: -0.1700

Data Index: 7067625 (Open Web Text)

Max Activating Token Index: 15

Text #14

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0605. Min Act: -0.1700

Data Index: 2787726 (Open Web Text)

Max Activating Token Index: 225

Text #15

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0532. Min Act: -0.1700

Data Index: 8136542 (Open Web Text)

Max Activating Token Index: 390

Text #16

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0259. Min Act: -0.1700

Data Index: 689646 (Open Web Text)

Max Activating Token Index: 695

Text #17

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0587. Min Act: -0.1700

Data Index: 1329667 (Open Web Text)

Max Activating Token Index: 71

Text #18

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0352. Min Act: -0.1700

Data Index: 7099185 (Open Web Text)

Max Activating Token Index: 490

Text #19

Max Range: 6.4912. Min Range: -6.4912

Max Act: 6.0416. Min Act: -0.1700

Data Index: 2066770 (Open Web Text)

Max Activating Token Index: 863