Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 108 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.6384. Min Act: -0.1700

Data Index: 1418818 (Python Code)

Max Activating Token Index: 611

Text #1

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.5677. Min Act: -0.1700

Data Index: 1396581 (Python Code)

Max Activating Token Index: 751

Text #2

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.4855. Min Act: -0.1700

Data Index: 1557978 (Python Code)

Max Activating Token Index: 491

Text #3

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.4010. Min Act: -0.1700

Data Index: 1536360 (Python Code)

Max Activating Token Index: 56

Text #4

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3735. Min Act: -0.1700

Data Index: 1393746 (Python Code)

Max Activating Token Index: 399

Text #5

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3516. Min Act: -0.1700

Data Index: 1462508 (Python Code)

Max Activating Token Index: 171

Text #6

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3580. Min Act: -0.1700

Data Index: 1564450 (Python Code)

Max Activating Token Index: 427

Text #7

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3277. Min Act: -0.1700

Data Index: 1390130 (Python Code)

Max Activating Token Index: 767

Text #8

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3100. Min Act: -0.1700

Data Index: 1403153 (Python Code)

Max Activating Token Index: 290

Text #9

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2544. Min Act: -0.1700

Data Index: 1402480 (Python Code)

Max Activating Token Index: 234

Text #10

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2864. Min Act: -0.1700

Data Index: 1371200 (Python Code)

Max Activating Token Index: 423

Text #11

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.3098. Min Act: -0.1700

Data Index: 1587806 (Python Code)

Max Activating Token Index: 914

Text #12

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2568. Min Act: -0.1700

Data Index: 1400594 (Python Code)

Max Activating Token Index: 241

Text #13

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2708. Min Act: -0.1700

Data Index: 1471158 (Python Code)

Max Activating Token Index: 740

Text #14

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2261. Min Act: -0.1700

Data Index: 1400854 (Python Code)

Max Activating Token Index: 823

Text #15

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2363. Min Act: -0.1700

Data Index: 1574372 (Python Code)

Max Activating Token Index: 367

Text #16

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2496. Min Act: -0.1700

Data Index: 1522186 (Python Code)

Max Activating Token Index: 908

Text #17

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2609. Min Act: -0.1700

Data Index: 1618998 (Python Code)

Max Activating Token Index: 759

Text #18

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2296. Min Act: -0.1700

Data Index: 1536933 (Python Code)

Max Activating Token Index: 276

Text #19

Max Range: 6.6384. Min Range: -6.6384

Max Act: 6.2402. Min Act: -0.1700

Data Index: 1592821 (Python Code)

Max Activating Token Index: 546