Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 564 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.3183. Min Act: -0.1700

Data Index: 3463178 (Open Web Text)

Max Activating Token Index: 290

Text #1

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.2538. Min Act: -0.1700

Data Index: 3503385 (Open Web Text)

Max Activating Token Index: 595

Text #2

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.2921. Min Act: -0.1700

Data Index: 7037222 (Open Web Text)

Max Activating Token Index: 663

Text #3

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.2672. Min Act: -0.1700

Data Index: 1506964 (Open Web Text)

Max Activating Token Index: 349

Text #4

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.2033. Min Act: -0.1700

Data Index: 356815 (Open Web Text)

Max Activating Token Index: 478

Text #5

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.2289. Min Act: -0.1700

Data Index: 3589791 (Open Web Text)

Max Activating Token Index: 918

Text #6

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1614. Min Act: -0.1700

Data Index: 872677 (Open Web Text)

Max Activating Token Index: 368

Text #7

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1761. Min Act: -0.1700

Data Index: 3813984 (Open Web Text)

Max Activating Token Index: 554

Text #8

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1304. Min Act: -0.1700

Data Index: 3329237 (Open Web Text)

Max Activating Token Index: 596

Text #9

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1088. Min Act: -0.1700

Data Index: 8059457 (Open Web Text)

Max Activating Token Index: 1022

Text #10

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1210. Min Act: -0.1700

Data Index: 8784095 (Open Web Text)

Max Activating Token Index: 374

Text #11

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1070. Min Act: -0.1700

Data Index: 3373811 (Open Web Text)

Max Activating Token Index: 841

Text #12

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0764. Min Act: -0.1700

Data Index: 3983307 (Open Web Text)

Max Activating Token Index: 481

Text #13

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1092. Min Act: -0.1700

Data Index: 3131246 (Open Web Text)

Max Activating Token Index: 369

Text #14

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0423. Min Act: -0.1700

Data Index: 8407881 (Open Web Text)

Max Activating Token Index: 1017

Text #15

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0606. Min Act: -0.1700

Data Index: 5812826 (Open Web Text)

Max Activating Token Index: 729

Text #16

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0924. Min Act: -0.1700

Data Index: 4608652 (Open Web Text)

Max Activating Token Index: 994

Text #17

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.1074. Min Act: -0.1700

Data Index: 7376941 (Open Web Text)

Max Activating Token Index: 184

Text #18

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0581. Min Act: -0.1700

Data Index: 530658 (Open Web Text)

Max Activating Token Index: 519

Text #19

Max Range: 5.3183. Min Range: -5.3183

Max Act: 5.0994. Min Act: -0.1700

Data Index: 5166338 (Open Web Text)

Max Activating Token Index: 610