Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 120 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0789. Min Act: -0.1700

Data Index: 2114641 (Open Web Text)

Max Activating Token Index: 98

Text #1

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0581. Min Act: -0.1700

Data Index: 2434907 (Open Web Text)

Max Activating Token Index: 940

Text #2

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0380. Min Act: -0.1700

Data Index: 8568369 (Open Web Text)

Max Activating Token Index: 528

Text #3

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0366. Min Act: -0.1700

Data Index: 5506907 (Open Web Text)

Max Activating Token Index: 827

Text #4

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0137. Min Act: -0.1700

Data Index: 4032499 (Open Web Text)

Max Activating Token Index: 146

Text #5

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0109. Min Act: -0.1700

Data Index: 477890 (Open Web Text)

Max Activating Token Index: 518

Text #6

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0077. Min Act: -0.1700

Data Index: 5594011 (Open Web Text)

Max Activating Token Index: 989

Text #7

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0223. Min Act: -0.1700

Data Index: 5877106 (Open Web Text)

Max Activating Token Index: 908

Text #8

Max Range: 3.0789. Min Range: -3.0789

Max Act: 3.0136. Min Act: -0.1700

Data Index: 7350895 (Open Web Text)

Max Activating Token Index: 457

Text #9

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9870. Min Act: -0.1700

Data Index: 7810267 (Open Web Text)

Max Activating Token Index: 345

Text #10

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9822. Min Act: -0.1700

Data Index: 6874789 (Open Web Text)

Max Activating Token Index: 643

Text #11

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9827. Min Act: -0.1700

Data Index: 3588320 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9916. Min Act: -0.1700

Data Index: 7350894 (Open Web Text)

Max Activating Token Index: 687

Text #13

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9742. Min Act: -0.1700

Data Index: 4960360 (Open Web Text)

Max Activating Token Index: 502

Text #14

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9737. Min Act: -0.1700

Data Index: 6620339 (Open Web Text)

Max Activating Token Index: 616

Text #15

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9646. Min Act: -0.1700

Data Index: 7383518 (Open Web Text)

Max Activating Token Index: 118

Text #16

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9645. Min Act: -0.1700

Data Index: 8202043 (Open Web Text)

Max Activating Token Index: 1000

Text #17

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9821. Min Act: -0.1700

Data Index: 8544393 (Open Web Text)

Max Activating Token Index: 828

Text #18

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9513. Min Act: -0.1700

Data Index: 3872299 (Open Web Text)

Max Activating Token Index: 655

Text #19

Max Range: 3.0789. Min Range: -3.0789

Max Act: 2.9630. Min Act: -0.1700

Data Index: 2029617 (Open Web Text)

Max Activating Token Index: 989