Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1305 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.9188. Min Act: -0.1700

Data Index: 8135159 (Open Web Text)

Max Activating Token Index: 15

Text #1

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.8739. Min Act: -0.1700

Data Index: 3048861 (Open Web Text)

Max Activating Token Index: 420

Text #2

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.8510. Min Act: -0.1700

Data Index: 2456297 (Open Web Text)

Max Activating Token Index: 333

Text #3

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.7191. Min Act: -0.1700

Data Index: 8308384 (Open Web Text)

Max Activating Token Index: 91

Text #4

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.7315. Min Act: -0.1700

Data Index: 8619175 (Open Web Text)

Max Activating Token Index: 569

Text #5

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6965. Min Act: -0.1700

Data Index: 7889333 (Open Web Text)

Max Activating Token Index: 113

Text #6

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.7180. Min Act: -0.1700

Data Index: 7680996 (Open Web Text)

Max Activating Token Index: 20

Text #7

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6507. Min Act: -0.1700

Data Index: 2369702 (Open Web Text)

Max Activating Token Index: 309

Text #8

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6714. Min Act: -0.1700

Data Index: 5329881 (Open Web Text)

Max Activating Token Index: 573

Text #9

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6731. Min Act: -0.1700

Data Index: 3986034 (Open Web Text)

Max Activating Token Index: 498

Text #10

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6442. Min Act: -0.1700

Data Index: 947582 (Open Web Text)

Max Activating Token Index: 39

Text #11

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6389. Min Act: -0.1700

Data Index: 5274677 (Open Web Text)

Max Activating Token Index: 196

Text #12

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6370. Min Act: -0.1700

Data Index: 1915647 (Open Web Text)

Max Activating Token Index: 733

Text #13

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6106. Min Act: -0.1700

Data Index: 3201852 (Open Web Text)

Max Activating Token Index: 897

Text #14

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.6000. Min Act: -0.1700

Data Index: 150473 (Open Web Text)

Max Activating Token Index: 87

Text #15

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.5803. Min Act: -0.1700

Data Index: 7976737 (Open Web Text)

Max Activating Token Index: 753

Text #16

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.5622. Min Act: -0.1700

Data Index: 1952904 (Open Web Text)

Max Activating Token Index: 422

Text #17

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.5985. Min Act: -0.1700

Data Index: 5102579 (Open Web Text)

Max Activating Token Index: 1003

Text #18

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.5618. Min Act: -0.1700

Data Index: 5015571 (Open Web Text)

Max Activating Token Index: 753

Text #19

Max Range: 4.9188. Min Range: -4.9188

Max Act: 4.5727. Min Act: -0.1700

Data Index: 6337897 (Open Web Text)

Max Activating Token Index: 65