Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 912 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.2201. Min Act: -0.1700

Data Index: 3309730 (Open Web Text)

Max Activating Token Index: 95

Text #1

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1870. Min Act: -0.1700

Data Index: 2859091 (Open Web Text)

Max Activating Token Index: 137

Text #2

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1972. Min Act: -0.1700

Data Index: 4104453 (Open Web Text)

Max Activating Token Index: 138

Text #3

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1981. Min Act: -0.1700

Data Index: 3998016 (Open Web Text)

Max Activating Token Index: 464

Text #4

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1501. Min Act: -0.1700

Data Index: 1172892 (Open Web Text)

Max Activating Token Index: 1017

Text #5

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1764. Min Act: -0.1700

Data Index: 2677454 (Open Web Text)

Max Activating Token Index: 208

Text #6

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1670. Min Act: -0.1700

Data Index: 3765114 (Open Web Text)

Max Activating Token Index: 95

Text #7

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1612. Min Act: -0.1700

Data Index: 7972755 (Open Web Text)

Max Activating Token Index: 253

Text #8

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1540. Min Act: -0.1700

Data Index: 6819392 (Open Web Text)

Max Activating Token Index: 138

Text #9

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1243. Min Act: -0.1700

Data Index: 7663124 (Open Web Text)

Max Activating Token Index: 987

Text #10

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1508. Min Act: -0.1700

Data Index: 5132504 (Open Web Text)

Max Activating Token Index: 492

Text #11

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1473. Min Act: -0.1700

Data Index: 4725528 (Open Web Text)

Max Activating Token Index: 205

Text #12

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1365. Min Act: -0.1700

Data Index: 7364474 (Open Web Text)

Max Activating Token Index: 338

Text #13

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1741. Min Act: -0.1700

Data Index: 726668 (Open Web Text)

Max Activating Token Index: 519

Text #14

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1263. Min Act: -0.1700

Data Index: 5308468 (Open Web Text)

Max Activating Token Index: 450

Text #15

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1346. Min Act: -0.1700

Data Index: 555331 (Open Web Text)

Max Activating Token Index: 129

Text #16

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1644. Min Act: -0.1700

Data Index: 2145012 (Open Web Text)

Max Activating Token Index: 409

Text #17

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1036. Min Act: -0.1700

Data Index: 4725525 (Open Web Text)

Max Activating Token Index: 618

Text #18

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1306. Min Act: -0.1700

Data Index: 1973741 (Open Web Text)

Max Activating Token Index: 420

Text #19

Max Range: 3.2201. Min Range: -3.2201

Max Act: 3.1159. Min Act: -0.1700

Data Index: 3309729 (Open Web Text)

Max Activating Token Index: 814