Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1464 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.3371. Min Act: -0.1700

Data Index: 4086330 (Open Web Text)

Max Activating Token Index: 776

Text #1

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.2572. Min Act: -0.1700

Data Index: 4951731 (Open Web Text)

Max Activating Token Index: 344

Text #2

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.2069. Min Act: -0.1700

Data Index: 2425627 (Open Web Text)

Max Activating Token Index: 161

Text #3

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1774. Min Act: -0.1700

Data Index: 4558924 (Open Web Text)

Max Activating Token Index: 936

Text #4

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.2063. Min Act: -0.1700

Data Index: 4636619 (Open Web Text)

Max Activating Token Index: 350

Text #5

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1645. Min Act: -0.1700

Data Index: 6416161 (Open Web Text)

Max Activating Token Index: 728

Text #6

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1524. Min Act: -0.1700

Data Index: 4662770 (Open Web Text)

Max Activating Token Index: 677

Text #7

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1524. Min Act: -0.1700

Data Index: 6177707 (Open Web Text)

Max Activating Token Index: 886

Text #8

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1588. Min Act: -0.1700

Data Index: 4179240 (Open Web Text)

Max Activating Token Index: 401

Text #9

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1522. Min Act: -0.1700

Data Index: 3301441 (Open Web Text)

Max Activating Token Index: 984

Text #10

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1733. Min Act: -0.1700

Data Index: 6162883 (Open Web Text)

Max Activating Token Index: 363

Text #11

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1213. Min Act: -0.1700

Data Index: 4041761 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1159. Min Act: -0.1700

Data Index: 3605453 (Open Web Text)

Max Activating Token Index: 866

Text #13

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1144. Min Act: -0.1700

Data Index: 5771203 (Open Web Text)

Max Activating Token Index: 502

Text #14

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.0781. Min Act: -0.1700

Data Index: 395922 (Open Web Text)

Max Activating Token Index: 571

Text #15

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1104. Min Act: -0.1700

Data Index: 2370353 (Open Web Text)

Max Activating Token Index: 850

Text #16

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.0959. Min Act: -0.1700

Data Index: 2952215 (Open Web Text)

Max Activating Token Index: 842

Text #17

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.0889. Min Act: -0.1700

Data Index: 3395829 (Open Web Text)

Max Activating Token Index: 268

Text #18

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.0968. Min Act: -0.1700

Data Index: 1131919 (Open Web Text)

Max Activating Token Index: 417

Text #19

Max Range: 4.3371. Min Range: -4.3371

Max Act: 4.1045. Min Act: -0.1700

Data Index: 3605035 (Open Web Text)

Max Activating Token Index: 355