Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1832 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.3048. Min Act: -0.1700

Data Index: 5438104 (Open Web Text)

Max Activating Token Index: 87

Text #1

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.2121. Min Act: -0.1700

Data Index: 8555723 (Open Web Text)

Max Activating Token Index: 900

Text #2

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.1727. Min Act: -0.1700

Data Index: 4809310 (Open Web Text)

Max Activating Token Index: 747

Text #3

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.1542. Min Act: -0.1700

Data Index: 5541049 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.1341. Min Act: -0.1700

Data Index: 6689530 (Open Web Text)

Max Activating Token Index: 211

Text #5

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.1253. Min Act: -0.1700

Data Index: 207695 (Open Web Text)

Max Activating Token Index: 423

Text #6

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.1313. Min Act: -0.1700

Data Index: 4227430 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0997. Min Act: -0.1700

Data Index: 1188398 (Open Web Text)

Max Activating Token Index: 757

Text #8

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0720. Min Act: -0.1700

Data Index: 7084091 (Open Web Text)

Max Activating Token Index: 382

Text #9

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0571. Min Act: -0.1700

Data Index: 231009 (Open Web Text)

Max Activating Token Index: 283

Text #10

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0840. Min Act: -0.1700

Data Index: 4230357 (Open Web Text)

Max Activating Token Index: 845

Text #11

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0750. Min Act: -0.1700

Data Index: 6133765 (Open Web Text)

Max Activating Token Index: 859

Text #12

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0722. Min Act: -0.1700

Data Index: 5267916 (Open Web Text)

Max Activating Token Index: 485

Text #13

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0557. Min Act: -0.1700

Data Index: 2102892 (Open Web Text)

Max Activating Token Index: 758

Text #14

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0267. Min Act: -0.1700

Data Index: 3246498 (Open Web Text)

Max Activating Token Index: 366

Text #15

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0235. Min Act: -0.1700

Data Index: 231257 (Open Web Text)

Max Activating Token Index: 437

Text #16

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0017. Min Act: -0.1700

Data Index: 2559656 (Open Web Text)

Max Activating Token Index: 808

Text #17

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0216. Min Act: -0.1700

Data Index: 4565360 (Open Web Text)

Max Activating Token Index: 847

Text #18

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0329. Min Act: -0.1700

Data Index: 4038806 (Open Web Text)

Max Activating Token Index: 291

Text #19

Max Range: 3.3048. Min Range: -3.3048

Max Act: 3.0224. Min Act: -0.1700

Data Index: 7310170 (Open Web Text)

Max Activating Token Index: 278