Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1362 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0432. Min Range: -5.0432

Max Act: 5.0432. Min Act: -0.1700

Data Index: 8812213 (Open Web Text)

Max Activating Token Index: 60

Text #1

Max Range: 5.0432. Min Range: -5.0432

Max Act: 5.0338. Min Act: -0.1700

Data Index: 5297763 (Open Web Text)

Max Activating Token Index: 514

Text #2

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9963. Min Act: -0.1700

Data Index: 6934390 (Open Web Text)

Max Activating Token Index: 902

Text #3

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9781. Min Act: -0.1700

Data Index: 1254841 (Open Web Text)

Max Activating Token Index: 367

Text #4

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9225. Min Act: -0.1700

Data Index: 60644 (Open Web Text)

Max Activating Token Index: 487

Text #5

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9343. Min Act: -0.1700

Data Index: 7462093 (Open Web Text)

Max Activating Token Index: 428

Text #6

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8777. Min Act: -0.1700

Data Index: 4316201 (Open Web Text)

Max Activating Token Index: 579

Text #7

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8663. Min Act: -0.1700

Data Index: 131333 (Open Web Text)

Max Activating Token Index: 994

Text #8

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8239. Min Act: -0.1700

Data Index: 7199334 (Open Web Text)

Max Activating Token Index: 582

Text #9

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8339. Min Act: -0.1700

Data Index: 2244777 (Open Web Text)

Max Activating Token Index: 838

Text #10

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8098. Min Act: -0.1700

Data Index: 1392359 (Open Web Text)

Max Activating Token Index: 425

Text #11

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8137. Min Act: -0.1700

Data Index: 412293 (Open Web Text)

Max Activating Token Index: 54

Text #12

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7636. Min Act: -0.1700

Data Index: 6292239 (Open Web Text)

Max Activating Token Index: 840

Text #13

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7735. Min Act: -0.1700

Data Index: 1954883 (Open Web Text)

Max Activating Token Index: 943

Text #14

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7753. Min Act: -0.1700

Data Index: 2028055 (Open Web Text)

Max Activating Token Index: 326

Text #15

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7699. Min Act: -0.1700

Data Index: 7454261 (Open Web Text)

Max Activating Token Index: 544

Text #16

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7692. Min Act: -0.1700

Data Index: 1650975 (Open Web Text)

Max Activating Token Index: 878

Text #17

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7815. Min Act: -0.1700

Data Index: 8666478 (Open Web Text)

Max Activating Token Index: 972

Text #18

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7437. Min Act: -0.1700

Data Index: 5349406 (Open Web Text)

Max Activating Token Index: 658

Text #19

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.7343. Min Act: -0.1700

Data Index: 3015856 (Open Web Text)

Max Activating Token Index: 192