Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2301 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.1344. Min Act: -0.1700

Data Index: 1925461 (Open Web Text)

Max Activating Token Index: 389

Text #1

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0550. Min Act: -0.1700

Data Index: 3441064 (Open Web Text)

Max Activating Token Index: 490

Text #2

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.1112. Min Act: -0.1700

Data Index: 7966455 (Open Web Text)

Max Activating Token Index: 406

Text #3

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0622. Min Act: -0.1700

Data Index: 1492507 (Open Web Text)

Max Activating Token Index: 503

Text #4

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0339. Min Act: -0.1700

Data Index: 3296810 (Open Web Text)

Max Activating Token Index: 362

Text #5

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9889. Min Act: -0.1700

Data Index: 5134291 (Open Web Text)

Max Activating Token Index: 801

Text #6

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0067. Min Act: -0.1700

Data Index: 5288524 (Open Web Text)

Max Activating Token Index: 421

Text #7

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0450. Min Act: -0.1700

Data Index: 4591599 (Open Web Text)

Max Activating Token Index: 417

Text #8

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0512. Min Act: -0.1700

Data Index: 1178246 (Open Web Text)

Max Activating Token Index: 392

Text #9

Max Range: 6.1344. Min Range: -6.1344

Max Act: 6.0202. Min Act: -0.1700

Data Index: 8030759 (Open Web Text)

Max Activating Token Index: 408

Text #10

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9895. Min Act: -0.1700

Data Index: 856910 (Open Web Text)

Max Activating Token Index: 432

Text #11

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9586. Min Act: -0.1700

Data Index: 2511386 (Open Web Text)

Max Activating Token Index: 463

Text #12

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9746. Min Act: -0.1700

Data Index: 7982365 (Open Web Text)

Max Activating Token Index: 407

Text #13

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9190. Min Act: -0.1700

Data Index: 2811838 (Open Web Text)

Max Activating Token Index: 582

Text #14

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9765. Min Act: -0.1700

Data Index: 4177419 (Open Web Text)

Max Activating Token Index: 446

Text #15

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9000. Min Act: -0.1700

Data Index: 3658936 (Open Web Text)

Max Activating Token Index: 840

Text #16

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9779. Min Act: -0.1700

Data Index: 1375040 (Open Web Text)

Max Activating Token Index: 711

Text #17

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.8925. Min Act: -0.1700

Data Index: 7944308 (Open Web Text)

Max Activating Token Index: 770

Text #18

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9603. Min Act: -0.1700

Data Index: 4961711 (Open Web Text)

Max Activating Token Index: 387

Text #19

Max Range: 6.1344. Min Range: -6.1344

Max Act: 5.9121. Min Act: -0.1700

Data Index: 1391811 (Open Web Text)

Max Activating Token Index: 561