Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1079 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.7293. Min Act: -0.0825

Data Index: 2145165 (Open Web Text)

Max Activating Token Index: 513

Text #1

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6762. Min Act: -0.0875

Data Index: 25513 (Open Web Text)

Max Activating Token Index: 953

Text #2

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6668. Min Act: 0.0022

Data Index: 1692453 (Open Web Text)

Max Activating Token Index: 546

Text #3

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6669. Min Act: -0.0909

Data Index: 2070878 (Open Web Text)

Max Activating Token Index: 340

Text #4

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6671. Min Act: -0.0253

Data Index: 6329904 (Open Web Text)

Max Activating Token Index: 842

Text #5

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6661. Min Act: -0.0880

Data Index: 1657575 (Open Web Text)

Max Activating Token Index: 448

Text #6

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6561. Min Act: 0.0287

Data Index: 8043931 (Open Web Text)

Max Activating Token Index: 212

Text #7

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6510. Min Act: -0.0138

Data Index: 287173 (Open Web Text)

Max Activating Token Index: 503

Text #8

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6536. Min Act: -0.1041

Data Index: 7556876 (Open Web Text)

Max Activating Token Index: 709

Text #9

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6502. Min Act: -0.0687

Data Index: 232771 (Open Web Text)

Max Activating Token Index: 643

Text #10

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6453. Min Act: -0.0482

Data Index: 1700091 (Open Web Text)

Max Activating Token Index: 579

Text #11

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6495. Min Act: -0.0368

Data Index: 6982383 (Open Web Text)

Max Activating Token Index: 350

Text #12

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6458. Min Act: -0.0779

Data Index: 7011653 (Open Web Text)

Max Activating Token Index: 946

Text #13

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6479. Min Act: -0.0426

Data Index: 5413673 (Open Web Text)

Max Activating Token Index: 485

Text #14

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6373. Min Act: -0.0760

Data Index: 1823272 (Open Web Text)

Max Activating Token Index: 601

Text #15

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6400. Min Act: -0.0049

Data Index: 2070876 (Open Web Text)

Max Activating Token Index: 704

Text #16

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6388. Min Act: -0.0569

Data Index: 1592651 (Open Web Text)

Max Activating Token Index: 702

Text #17

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6400. Min Act: -0.0738

Data Index: 6657830 (Open Web Text)

Max Activating Token Index: 369

Text #18

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6348. Min Act: -0.0598

Data Index: 6285691 (Open Web Text)

Max Activating Token Index: 579

Text #19

Max Range: 1.7293. Min Range: -1.7293

Max Act: 1.6395. Min Act: -0.1185

Data Index: 1537414 (Open Web Text)

Max Activating Token Index: 336