Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 224 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0026. Min Range: -6.0026

Max Act: 6.0026. Min Act: -0.1700

Data Index: 2299504 (Open Web Text)

Max Activating Token Index: 449

Text #1

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.9001. Min Act: -0.1700

Data Index: 292823 (Open Web Text)

Max Activating Token Index: 517

Text #2

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.8650. Min Act: -0.1700

Data Index: 5120720 (Open Web Text)

Max Activating Token Index: 179

Text #3

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.8143. Min Act: -0.1700

Data Index: 7753454 (Open Web Text)

Max Activating Token Index: 714

Text #4

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.7988. Min Act: -0.1700

Data Index: 4636241 (Open Web Text)

Max Activating Token Index: 766

Text #5

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.7700. Min Act: -0.1700

Data Index: 7686926 (Open Web Text)

Max Activating Token Index: 1006

Text #6

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.7041. Min Act: -0.1700

Data Index: 7012273 (Open Web Text)

Max Activating Token Index: 633

Text #7

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6775. Min Act: -0.1700

Data Index: 285388 (Open Web Text)

Max Activating Token Index: 358

Text #8

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6896. Min Act: -0.1700

Data Index: 1994966 (Open Web Text)

Max Activating Token Index: 519

Text #9

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6324. Min Act: -0.1700

Data Index: 3248549 (Open Web Text)

Max Activating Token Index: 694

Text #10

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6524. Min Act: -0.1700

Data Index: 7440722 (Open Web Text)

Max Activating Token Index: 482

Text #11

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6104. Min Act: -0.1700

Data Index: 6696693 (Open Web Text)

Max Activating Token Index: 771

Text #12

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6264. Min Act: -0.1700

Data Index: 7951423 (Open Web Text)

Max Activating Token Index: 744

Text #13

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6293. Min Act: -0.1700

Data Index: 5216129 (Open Web Text)

Max Activating Token Index: 542

Text #14

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6131. Min Act: -0.1700

Data Index: 3494325 (Open Web Text)

Max Activating Token Index: 281

Text #15

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6262. Min Act: -0.1700

Data Index: 2474037 (Open Web Text)

Max Activating Token Index: 553

Text #16

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.6034. Min Act: -0.1700

Data Index: 6832535 (Open Web Text)

Max Activating Token Index: 386

Text #17

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.5844. Min Act: -0.1700

Data Index: 7627927 (Open Web Text)

Max Activating Token Index: 630

Text #18

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.5802. Min Act: -0.1700

Data Index: 1617607 (Open Web Text)

Max Activating Token Index: 584

Text #19

Max Range: 6.0026. Min Range: -6.0026

Max Act: 5.5642. Min Act: -0.1700

Data Index: 2410119 (Open Web Text)

Max Activating Token Index: 840