Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2613 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.4778. Min Act: -0.1700

Data Index: 1903073 (Open Web Text)

Max Activating Token Index: 351

Text #1

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.3999. Min Act: -0.1700

Data Index: 7553936 (Open Web Text)

Max Activating Token Index: 91

Text #2

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.3783. Min Act: -0.1700

Data Index: 5762315 (Open Web Text)

Max Activating Token Index: 199

Text #3

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.3325. Min Act: -0.1700

Data Index: 679039 (Open Web Text)

Max Activating Token Index: 707

Text #4

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.2373. Min Act: -0.1700

Data Index: 6951924 (Open Web Text)

Max Activating Token Index: 354

Text #5

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.2514. Min Act: -0.1700

Data Index: 5047338 (Open Web Text)

Max Activating Token Index: 968

Text #6

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.2115. Min Act: -0.1700

Data Index: 7331940 (Open Web Text)

Max Activating Token Index: 546

Text #7

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.2135. Min Act: -0.1700

Data Index: 1770864 (Open Web Text)

Max Activating Token Index: 978

Text #8

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1706. Min Act: -0.1700

Data Index: 1026285 (Open Web Text)

Max Activating Token Index: 644

Text #9

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1654. Min Act: -0.1700

Data Index: 780282 (Open Web Text)

Max Activating Token Index: 940

Text #10

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1512. Min Act: -0.1700

Data Index: 5424543 (Open Web Text)

Max Activating Token Index: 699

Text #11

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1535. Min Act: -0.1700

Data Index: 1066656 (Open Web Text)

Max Activating Token Index: 812

Text #12

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1131. Min Act: -0.1700

Data Index: 7723366 (Open Web Text)

Max Activating Token Index: 412

Text #13

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1072. Min Act: -0.1700

Data Index: 7548893 (Open Web Text)

Max Activating Token Index: 851

Text #14

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.1013. Min Act: -0.1700

Data Index: 4271997 (Open Web Text)

Max Activating Token Index: 312

Text #15

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.0963. Min Act: -0.1700

Data Index: 76238 (Open Web Text)

Max Activating Token Index: 403

Text #16

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.0687. Min Act: -0.1700

Data Index: 5018231 (Open Web Text)

Max Activating Token Index: 667

Text #17

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.0758. Min Act: -0.1700

Data Index: 4627987 (Open Web Text)

Max Activating Token Index: 474

Text #18

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.0271. Min Act: -0.1700

Data Index: 1668615 (Open Web Text)

Max Activating Token Index: 429

Text #19

Max Range: 3.4778. Min Range: -3.4778

Max Act: 3.0170. Min Act: -0.1700

Data Index: 4332276 (Open Web Text)

Max Activating Token Index: 790