Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2746 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1041. Min Range: -4.1041

Max Act: 4.1041. Min Act: -0.1700

Data Index: 8478789 (Open Web Text)

Max Activating Token Index: 676

Text #1

Max Range: 4.1041. Min Range: -4.1041

Max Act: 4.0570. Min Act: -0.1700

Data Index: 7853863 (Open Web Text)

Max Activating Token Index: 371

Text #2

Max Range: 4.1041. Min Range: -4.1041

Max Act: 4.0543. Min Act: -0.1700

Data Index: 2560665 (Open Web Text)

Max Activating Token Index: 399

Text #3

Max Range: 4.1041. Min Range: -4.1041

Max Act: 4.0325. Min Act: -0.1700

Data Index: 1334413 (Open Web Text)

Max Activating Token Index: 15

Text #4

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.9921. Min Act: -0.1700

Data Index: 4642398 (Open Web Text)

Max Activating Token Index: 787

Text #5

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.9123. Min Act: -0.1700

Data Index: 3052266 (Open Web Text)

Max Activating Token Index: 54

Text #6

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.9041. Min Act: -0.1700

Data Index: 4330985 (Open Web Text)

Max Activating Token Index: 412

Text #7

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.9141. Min Act: -0.1700

Data Index: 2069950 (Open Web Text)

Max Activating Token Index: 675

Text #8

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8886. Min Act: -0.1700

Data Index: 3277465 (Open Web Text)

Max Activating Token Index: 182

Text #9

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8928. Min Act: -0.1700

Data Index: 2503275 (Open Web Text)

Max Activating Token Index: 37

Text #10

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8796. Min Act: -0.1700

Data Index: 6180536 (Open Web Text)

Max Activating Token Index: 861

Text #11

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8804. Min Act: -0.1700

Data Index: 7771622 (Open Web Text)

Max Activating Token Index: 382

Text #12

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8773. Min Act: -0.1700

Data Index: 2351730 (Open Web Text)

Max Activating Token Index: 845

Text #13

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8962. Min Act: -0.1700

Data Index: 942579 (Open Web Text)

Max Activating Token Index: 651

Text #14

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8914. Min Act: -0.1700

Data Index: 26682 (Open Web Text)

Max Activating Token Index: 473

Text #15

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8889. Min Act: -0.1700

Data Index: 4260633 (Open Web Text)

Max Activating Token Index: 678

Text #16

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8922. Min Act: -0.1700

Data Index: 1484469 (Open Web Text)

Max Activating Token Index: 965

Text #17

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8898. Min Act: -0.1700

Data Index: 6018754 (Open Web Text)

Max Activating Token Index: 986

Text #18

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8669. Min Act: -0.1700

Data Index: 895801 (Open Web Text)

Max Activating Token Index: 828

Text #19

Max Range: 4.1041. Min Range: -4.1041

Max Act: 3.8537. Min Act: -0.1700

Data Index: 1883298 (Open Web Text)

Max Activating Token Index: 914